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VOLUME I: TELEPRESENCE TECHNOLOGY BASE DEVELOPMENT 


1.1 INTRODUCTION 

1.1.1 CONTRACTUAL BACKGROUND OF STUDY 

On June 10, 1982, NASA Marshall Space Flight Center (MSFC) awarded a 
twelve month contract (NAS8“34381) to the Space Systems and the Artificial 
Intelligence Laboratories of the Massachusetts Institute of Technology, for a 
study entitled "Space Applications of Automation, Robotics, and Machine 
Intelligence Systems (ARAMIS), Phase II, Telepresence". This Phase 11 contract 
immediately followed the completion of the ARAMIS Phase I research (also 
contract NAS8~34381) which produced its own final report. The Space Systems 
Laboratory is part of the MIT Department of Aeronautics and Astronautics; the 
Artificial Intelligence Laboratory is one of MIT^s interdepartmental 
1 aborator i es. Work on the contract began on June 10, 1981> with a termination 
date for Phase I I on June 9, 1983* 

This document is the final report for Phase II of the ARAMIS study. The 
NASA MSFC Contracting Officer's Representative is Georg F. von Tiesenhausen 
(205-ii53-2789) . 

1.1.2 CONTRIBUTORS TO THIS STUDY 

The members of the study team are listed in Table 1.1. Information 
necessary for this study was obtained from experts in government, industry, and 
academia, and from literature searches. 

Principal Investigators; 

Professor David L. Akin (6 17“253~3626) 

Professor Marvin L. Minsky (617“253“586A) 

Study Manager: Eric D. Thiel (617“253“2298) 

Associate Study Manager: Clifford R. Kurtzman (617-253-2298) 

Contributing Investigator: Professor Rene H. Miller (617-253-2263) 

Research Staff: 

Russell D. Howard 
Joseph S. Oliveira 

Part-time Researcher: Antonio Marra, Jr. 

TABLE 1.1: STUDY PARTICIPANTS 
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1.1.3 ORGANIZATION OF THE FINAL REPORT 


Volume 1 of this report is the Telepresence Technology Base Development. 
This volume defines the field of telepresence, and provides overviews of those 
capabilities that are now available, and those that will be required to support 
a NASA telepresence effort. This includes investigation of NASA's plans and 
goals with regard to telepresence, extensive literature search for materials 
relating to relevant technologies, a description of these technologies and 
their state-of-the-art, and projections for advances in these technologies over 
the next decade. Also included is a listing of facilities that are doing 
research and development relating to telepresence. A technology development 
program leading to the deployment of an operational telepresence system by 1992 
is presented. Volume 1 of this report is intended as a broad approach to 
telepresence technology and the general development of that technology. 

Volume 2 of this report is the Telepresence Project Applications. This 
volume examines several space projects in detail to determine what capabilities 
are required of a telepresence system in order to accomplish various tasks, 
such as servicing and assembly. The key operational and technological areas 
are identified, conclusions and recommendations are made for further research, 
and an example developmental program is presented, leading to an operational 
telepresence servicer. Volume 2 is intended as an example of telepresence 
technology, and the associated issues, when telepresence is applied to several 
specific space missions. 

Volume 3 is the executive summary of this contract report, 

1.1. TELEPRESENCE DESCRIPTION 

For the reader not familiar with telepresence, this section is Intended 
as a brief introduction to the concept of telepresence and some of the 
terminology used in this report. 
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Roughly translated, the word "telepresence'- means remote presence, just 
as "teleoperator" means remote operation. One way to think of telepresence is 
as a high fidelity teleoperator system. A teleoperator receives instructions 
from a human operator, and performs some action based on the instructions at a 
location remote from the human operator. It is similar to an industrial robot, 
except that a human is in control instead of a computer. 

The distinction between telepresence and teleoperation is in the 
capabilities of the manipulators, and the quality and quantity of information 
available to the operator. 


TELEPRESENCE DEFINITION 

AT THE WORKSITE, THE MANIPULATORS HAVE THE DEXTERITY TO 
ALLOW THE OPERATOR TO PERFORM NORMAL HUMAN FUNCTIONS 

AT THE CONTROL STATION, THE OPERATOR RECEIVES SUFFICIENT 
QUANTITY AND QUALITY OF SENSORY FEEDBACK TO PROVIDE A FEELING 
OF ACTUAL PRESENCE AT THE WORKSITE 

The operator uses motions similar to those which he/she would use at the 
worksite to control manipulators capable of accomplishing operations. The 
information available to the operator should maximize the feeling of being 
present at the worksite. This permits the operator to concentrate on the work 
using his/her natural abilities to perform the task, without being distracted 
by unnecessary differences between actually being present and using a remote 
system. 

The purpose of a telepresence system is to perform space operations which 
require human intelligence, control, and dexterity when EVA is not possible, 
not desirable, or when EVA alone cannot accomplish the desired mission. A 
telepresence system should permit remote assembly and repair of spacecraft. 
Also, it will permit unanticipated problems to be solved. Skylab, Apollo 13 » 
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and the planned repair of the Solar Max spacecraft all demonstrate the 
importance of human capabilities for solving problems. Fortunately, humans 
were onboard both Apollo and Skylab to perform repairs, and Solar Max is 
within EVA range, but failures will occur on spacecraft which are out of EVA 
range or time limits. Telepresence is a necessary part of future space 
operations. 

1.1.5 DEFINITIONS AND ' EXPLA I NAT I ONS 

The following set of definitions have been included to aid the reader in 
understanding this report. 

DEXTERITY. The more dexterous the system, the closer it is to being able 
to perform the same tasks as an onsite human. This does not mean that the 
dexterous system performs the tasks in the same manner as an onsite human, but 
that it can produce the same results. Dexterity does not require high 
precision. A human arm is certainly dexterous, but it cannot position itself 
with the precision of a non-dexterous robot arm. The human arm can do the same 
tasks because it is compliant and has excellent feedback in its control system. 

DOF. Degrees-of-Freedom. In the context of this report, this is the 
number of separate motions at each arm joint. For example, a manipulator with 
3 axis of rotation at the shoulder, one at the elbow, and three at the wrist 
has 7 DOF. 

END EFFECTOR. The '‘hand” portion of the manipulator arm. 

EVA. Extra-Vehicular-Activity. Astronauts in pressure suits working 
outside a spacecraft. EVA is presently limited to near shuttle operations and 
6 hours duration. Radiation exposure safety requirements will limit E.VA above 
shuttle altitudes to short duration missions. EVA technology and analysis used 
in this report are based upon present EVA pressure suit technology. 
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FORCE FEEDBACK. This term describes the process of providing the operator 
with knowledge of the amount of force he is applying with the manipulator. 

This may include torques as well as direct linear forces, but it is not 
intended as a sense of touch. 

NON-DEXTEROUS ARM. Often used to describe robot arms designed for one set 
of tasks. Implies an arm of limited capacity which is usually limited to a 
small number of specialized tasks. 

SERVICER. A spacecraft, probably using telepresence, that resupplies, 
repairs, or assembles another spacecraft. 

SUPERVISORY CONTROL. This is a control mode using a mix of man and 
machine control. In supervisory control the operator uses high level commands, 
as opposed to direct control, to instruct the computer to perform selected 
tasks. Upon completion of the desired task the computer returns control of the 
system to the operator. 

TACTILE FEEDBACK. This is the process of providing the operator with a 
sense of touch, although the information may be transmitted to the operator by 
visual display, or other means. The operator would not only be able to 
distinguish differential pressures across a sensing surface, but would also 
have the ability to sense parameters such as slip, texture, and/or temperature. 

TELEOPERATOR. Remotely controlled manipulator which to date has usually 
required direct human vision (no video system) for control. Generally used to 
refer to present and past remote manipulators. An example Is the manipulators 
used in the nuclear industry. 

TELEPRESENCE. The concept of telepresence can be defined by the following 
two statements. At the worksite, the manipulators have the dexterity to allow 
the operator to perform normal human functions. At the control station, the 
operator receives sensory feedback of sufficient quantity and quality to 
provide a feeling of actual presence at the worksite. These two statements 
represent the definition of an "all up" telepresence system. 
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TELEPRESENCE SERVICER. A telepresence unit performing spacecraft 
servicing tasks. 

TELEPRESENCE UNIT. The part of the telepresence system that is 
transported to the worksite to perform the desired tasks. 

The terms function, operation, and task are used interchangeably in this 
report. 

1.2 EXAMINATION OF NASA GOALS AND PLANS 
1.2.1 INTRODUCTION 

To determine the technology required for telepresence, the general tasks 
required of a telepresence system must first be understood. This volume 
considers NASA goals and plans in a general sense, both near and far term. 
Telepresence is summarized as the ability to perform certain broadly defined 
functions. The second volume of this report considers the application of 
telepresence to specific spacecraft programs, and examines the details and 
operational considerations of telepresence operations. The telepresence 
technology base (described in sections 1.3 and 1.3) is based upon the need to 
perform the telepresence functions developed in this section. 

NASALS plans can be divided into near term (through about 1995) and far 
term (post 1995)* There is necessarily some overlap between these divisions 
because of planning and scheduling uncertainties, but there is a clear 
difference in the levels of planning detail for these periods. Near term plans 
and goals are detailed enough to permit reasonable assumptions about missions 
and procedures; these assumptions are sufficient to determine technology 
requirements. Far term plans are not specific enough to permit a determination 
of technology requirements beyond identifying general areas of research 
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Any estimation of the proper technology to be used to solve a future 
problem will be heavily influenced by the available and currently projected 
technological capabilities in the problem area. Thus, the technology 
requirements in section 1.3 consider applicable any technology which could be 
developed, space qualified, and integrated into a space telepresence system 
which has an initial operational capability of 1990 to 1992. 

1.2.2 NEAR TERM GOALS AND PLANS 

The near terms goals and plans can be divided into three areas; spacecraft 
servicing, structural assembly, and contingency events. Figures 1.1, 1.2, and 
1.3 illustrate the three areas, and the class of operation to which they 
belong. Scheduled operations consist of any task which can be anticipated and 
has a predictable frequency. Unscheduled operations are also anticipated, but 
do not have a predictable frequency. Contingency operations are unanticipated 
events which were not considered in the spacecraft design or the mission 
planning. Thus, the use of EVA to close the orbiter payload bay doors Is an 
unscheduled operation; the repair of the Skylab solar panel was a contingency 
operat i on . 


1.2.2. 1 SPACECRAFT SERVICING 

Servicing is the most important area for near term telepresence 
application. NASA is firmly committed to servicing such spacecraft as Space 
Telescope, the Advanced X-ray Astronomy Facility (AXAF) , and the Long Duration 
Exposure Facility (LDEF) . In addition, the success of the Solar Max Mission 
depends on an EVA repair scheduled for STS 13> Also, servicing is virtually 
mandatory for large scale space processing of materials, for space stations, 
and for space operations in general. Such large scale projects may not be 
fully developed by 1995* but the technology must be developed and in place 
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Figure 1.3: Contingency Events. 



prior to full scale operations to provide servicing as needed. 

A key problem with servicing planning is the "inertia” in spacecraft 
design and future planning. This inertia is endemic in the aerospace industry, 
but is particularly noticeable in servicing plans. 

Essentially, the problem is that almost any servicing function can be 
performed with low level or near present technology, if the spacecraft is 
specifically designed to accommodate servicing performed by that type of 
technology. The end result is that servicing planning is currently limited to 
either simplistic module exchange devices or EVA operations. More advanced 
approaches (telepresence) are not being planned for because the technology is 
not being developed, and the technology is not being developed because there 
are no planned uses for it. This statement does not hold true for long term 
plans because some of the missions, by definition, require dexterous operations 
beyond EVA altitude and time capabilities, but telepresence capabilities will 
be needed prior to 2000. 

Using more advanced technology, such as telepresence, has several 
advantages over low level technology such as non-dexterous module exchange 
devices. In general, the more advanced the servicer, the less impact servicing 
will have on the spacecraft design. Also, the advanced technology increases 
the reliability and versatility of the entire system. Consider the case of a 
jammed module or servicer arm. A module exchange mechanism could do little to 
solve the problem, and could conceivably be unable to detach itself from the 
crippled spacecraft, thus rendering both Itself and the spacecraft useless. A 
more advanced teleoperator with two arms might be able to solve the problem; at 
the least it should be capable of freeing itself from the spacecraft. Such a 
system would also be capable of handling some contingency operations (see 
section 1 .2.2.3) • 

Spacecraft servicing is composed of both scheduled and unscheduled 
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operations. Many of the individual near term mechanical tasks (module 
exchanges, for example) can be performed by robot devices similar to today's 
industrial robots, but the ability to assemble these tasks into more complex 
operations, handle even small anomalies in behavior, or reach through a crowded 
workspace, requires human control, as is provided by telepresence. 

1.2. 2. 2 STRUCTURAL ASSEMBLY 

NASA's near term plans do not explicitly call for structural assembly, but 
operations of this kind will probably be used for space station and other 
pre-1995 missions. Also, a system capable of performing near term servicing 
tasks is probably capable of performing many structural assembly tasks. 

Most of the tasks required for structural assembly are simple positioning 
and manipulation operations, which should require less dexterity than servicing 
tasks. Some other capabilities are required for some assembly scenarios, such 
as cutting and welding, and can be accomplished with various tools or end 
effectors. 

Some structural assembly tasks are scheduled, but most would be 
unscheduled because of the complexity of the work environment, and because it 
is doubtful that everything will be in a predetermined location at a 
predetermined time. Structural assembly tasks cannot be performed by simple 
industrial robot devices, because assembly Is composed of many unscheduled 
operations which must be performed sequentially. The assembly of a structure 
requires the positioning and attachment of structural elements based on the 
assembly status of many other components In the structure. It is unlikely that 
a project of any size will exactly follow a preset construction plan; 
components will not always be exactly where they are expected nor behave in an 
expected manner. Also, the development of such an exact plan may be infeasible 
for many structures because ground simulations of space operations are not 
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completely reliable. A device similar to a present industrial robot would not 
be able to cope with such a flexible work environment. The unscheduled 
servicing operations are composed of a preplanned and predictable series of 
events. Unless performed by EVA, structural assembly needs a telepresence 
system and is, in many respects, more like contingency events than spacecraft 
servi ci ng. 

1.2. 2. 3 CONTINGENCY EVENTS 

Discussion with NASA representatives indicates that the ability to handle 
contingency events is a priority capability. An examination of the Sky lab or 
Apollo programs indicates that contingency operations have been of enormous 
benefit to the space program. 

Less dramatic reasons exist for a contingency capability. On-orbit 
failures of spacecraft will become more common as the space program transitions 
to a space industry. Contingency repairs, such as the Solar Max repair, will 
be a necessary part of our space operations. Other, more complicated or 
dangerous tasks (replacing a failed battery or fuel tank, rescue operations, 
etc.), may exceed the EVA operations envelope and require a teleoperator 
mission. A spacecraft which has stopped communicating would require either EVA 
or a teleoperator of some type to approach it and make a diagnosis. 

The contingency events discussed here may seem rather advanced for near 
term consideration, but they are possible contingency events which are, by 
definition, unplanned and unanticipated. Also, the repairs performed by a 
telepresence system would be determined by the details of the individual case 
and the technology. aval 1 abl e. An example of this Is the Sky lab program, in 
which the repair procedures developed were based on the capabilities and 
limitations of extra-vehicular assembly. 
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1.2.2.1* NEAR TERM TASK SUMMARY 


‘Table 1.2 is a listing of the basic tasks which the study group has 
developed. The tasks are meant to be representative of the activities which 
are necessary for NASA to accomplish its goals* particularly spacecraft 
servicing* but are not intended as an exhaustive list of possible telepresence 
capabilities. These tasks are used to develop the telepresence technology 
requirements presented in section 1.3> An advanced telepresence system would 
be capable of more intricate tasks than those listed in Table 1.2. 

OPERATE MECHANICAL CONNECTION 

OPERATE ELECTRICAL CONNECTION 

OPERATE LATCHING DEVICE 

GRASP OBJECT 

POSITION OBJECT 

OPERATE CUTTING DEVICE 

OPERATE WELDING DEVICE 

GRAPPLE DOCKING FIXTURE OR HANDHOLD 

OBSERVE SPACECRAFT/COMPONENT 

TABLE 1.2: TELEPRESENCE TASK SUMMARY 

These tasks are general in nature, and each could be either very simple or 
very complex. They are intended as a listing of basic mechanical operations* 
which can be combined to perform near term spacecraft servicing* structural 
assembly* and contingency events. 

A brief consideration of spacecraft design* and the necessary 
characteristics of any system capable of performing spacecraft servicing, 
indicates that remote manipulators similar to those used on the ground today 
could accomplish these tasks. They would be slower and exhibit more difficulty 
than would a human in a shirt sleeve environment, but they could perform the 
necessary operations. In summary, the near term requirements are fairly simple 
mechanical operations which are within the capabilities of present ground 
man i pulators . 
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1.2. 2. 5 EVA EQUIVALENT CAPABILITY 

A comparison of the tasks listed in Table 1.2 with past EVA operations and 
neutral buoyancy simulations for Space Telescope and other missions indicates 
that the tasks required for NASA's near term goals could all be accomplished by 
EVA. This is not surprising, since most servicing plans call for EVA to 
perform the servicing. However, a consideration of reasonable manipulator and 
servicer technologies also leads to combinations of simple mechanical 
operations, which are similar to EVA tasks. 

In addition to the fact that near term telepresence tasks are similar to 
EVA capabilities, there are several other justifications for designing near 
term telepresence systems to match EVA capabilities. NASA has experience with 
EVA operations, and this experience will continue to grow as the STS program 
continues. Since the Gemini program, NASA and industry have been accumulating 
design experience for EVA hardware and procedures. This experience is growing 
rapidly through programs such as Space Telescope, and efforts are being made to 
standardize spacecraft fittings and connections to facilitate space operations. 
This experience has produced confidence that EVA is capable of performing 
useful and important tasks. A telepresence system with capabilities similar to 
EVA would be able to utilize this experience in design and operations. It 
would also only need to demonstrate its ability to perform EVA tasks in one or 
two comprehensive tests to be considered capable of a wide variety of space 
tasks. A system with radically different capabilities than EVA would require 
more time and testing before confidence in its abilities could be established. 
Also, EVA and telepresence systems with similar capabilities would be capable 
of mutual backup operations and simultaneous operations. This would be 
especially useful during initial testing, and during very difficult operations. 
Furthermore, a telepresence system with an EVA equivalent capability would 


1 . 2.10 



provide for a smooth transition from our present technology of all EVA to a 
more advanced man-machine mix. Spacecraft designed for EVA or telepresence 
servicing would be serviceable by both methods. Spacecraft designed for EVA 
servicing would be only slightly different that those designed for telepresence 
servicing, due mostly to size and reach differences. This is not as important 
for non-LEO spacecraft because they are currently inaccessible to EVA, but near 
term servicing and assembly operations will be performed in LEO. 

Finally, EVA equivalency does, by definition, include the ability to 
perform simple contingency operations. 

It should be pointed out that the EVA equivalent capability does not mean 
that the telepresence system would perform the same tasks in the same manner as 
EVA. Telepresence might take longer, require more tools, and follow different 
procedures than EVA, but it would achieve the same results. Also, this EVA 
capability is based upon present suit technology. Future suit technology 
should significantly improve dexterity. Since both manipulator, end effector, 
and suit technologies are advancing, EVA and telepresence should continue to 
complement each other's operations through 2000. 

1 . 2.3 LONG TERM PLANS AND GOALS 

NASA's long term plans and goals are not specific or certain enough to 
permit definite conclusions other than general areas of interest. These areas 
of interest, or general goals, correspond closely with the potential future 
capabilities discussed in section 1.3*3 Advanced Technology. NASA will be able 
to utilize advanced technology, which is a natural product of present and near 
term research, to meet its long term goals. Unlike the technology necessary 
for near term telepresence, much of the advanced development will be performed 
by research in artificial intelligence and supervisory control which is not 
funded by NASA, although NASA support will be required to develop advanced A1 
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technologies for space use. 

The most important long term goals are increased system dexterity and the 
ability for contingency operations. As space operations become space industry, 
and the construction, modification, and repair of orbital systems becomes 
routine, onsite high dexterity manipulation will be mandatory. Equipment 
shipped from Earth will not be preassembled as it is today, but will arrive as 
spares and components for orbital construction and assembly. Some of the 
components will probably require high dexterity assembly. More importantly, 
the need to replace damaged and failed components, particularly in intricate 
mechanical devices or complex systems, will require dexterity simply to access 
the repair site. An example is the modification or repair of a wiring harness. 
Despite clever design and much effort, there will be places where wiring will 
need to be guided through a harness that is difficult to reach, and which 
requires hand dexterity to feed the wiring. 

The potential size and scope of future space operations will prohibit the 
extreme caution and highly detailed planning that accompanies present space 
missions. Commercial space missions will be commonplace, and industrial 
accidents will occur. The failure of a large materials processing furnace or a 
high pressure fuel line implies the need for crew rescue and versatile repair 
tasks. Tasks of this nature necessitate the ability to deal with nonfunctional 
and severely damaged equipment in an environment which may be unsuitable for 
EVA. The probability of successful advanced contingency operations is improved 
greatly by the availability of high dexterity telepresence. 

Driven partly by the scope of future operations and partly by the fact 
that transmission time delays may degrade dexterity, increased system autonomy 
is desirable. Many future tasks could be repetitive and boring: high level 
supervisory control for these tasks would relieve operator fatigue and improve 
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reliability. In regions of obscured communications, an autonomous operation 
capability is necessary. Transmission time delays may make remote high 
dexterity control difficult or impossible, so some otherwise mundane tasks 
could require supervisory control or autonomy. 

Due to the large costs of space vehicles, improvements to the telepresence 
system should be evolutionary, so that a new spacecraft is not required for 
each system upgrade. As spacecraft technology improves, the maneuvering system 
and telepresence unit may be replaced, but manipulator or computer system 
upgrades, for example, should not require replacing the entire spacecraft. The 
most radical advances in telepresence technology will occur in computer 
hardware and software, manipulators, and end effectors. Once a high dexterity 
manipulator is developed and installed, most system changes will be in 
software, which can be performed remotely from ground or space station control 
centers . 

1.2.4 TELEPRESENCE PLANS AND GOALS CONCLUSIONS 

For near term space operations, telepresence systems should be designed to 
be equivalent to EVA in capabilities. Telepresence may use different methods 
and may require more time to perform a given task than EVA, but telepresence 
should be able to achieve the same results. An EVA equivalent capability is 
desirable because it is more reliable than less capable options, such as the 
module exchange mechanism previously discussed, and is necessary for a minimum 
contingency operations capability. Also, an EVA equivalent telepresence system 
would have the option of using EVA as a backup and vice versa. Given the state 
of the technology discussed in section 1.3, an EVA equivalent telepresence 
system is a reasonable and timely development. 

Long term telepresence goals are increased dexterity and autonomy. A 
rapidly growing workload composed of increasingly complex tasks will require 
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high dexterity manipulators and end effectors. The potential size and scope of 
future space operations and the desire for advanced contingency operations 
indicate that autonomy is an important goal. 

1.3 TECHNOLOGY REQUIREMENTS AND ASSESSMENT 
1 . 3.1 INTRODUCTION 

This section presents the technology necessary to meet NASA's near term 
goals for telepresence. It also includes an assessment and discussion of the 
available technology. The requirements and assessments are organized by the 
technology areas shown in Figure 1.4* except for space adaptation, which is not 
a specific technology area, but is included in the figure due to the importance 
and difficulty of space qualifying hardware. Selected advanced features are 
included in the assessments where beneficial to system performance and 
operation. The assessments include recommended research and development 
topics, as well as state-of-the-art and promising near term technologies, A 
listing of present, near term, and advanced technologies is included in section 
1.5t Development Program. This listing should be examined along with the 
technology assessment, because it covers a much wider time frame, and provides 
a concise statement of present and future technologies. 

Section 1.3.3 presents a discussion of advanced telepresence technology. 
The advanced technology is also organized by technology areas, but is not 
presented in a requirements and assessments format, because NASA's long term 
plans are not specific enough to permit accurate development of technology 
requirements. Furthermore, technology forecasts past 1995 are of questionable 
accuracy. 

Some of the technology requirements for an initial operational 
telepresence system may seem overly advanced to some readers, but the study 
group believes that such a system could be developed and flown following the 
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Figure 1,4: Telepresence Technology Areas 


SPACE ADAPTATIO 













procedures and schedule presented In section 1 > 5 < Some options selected may 
not be absolutely necessary for a telepresence system operation, but have the 
potential to greatly improve performance or safety at minimal cost or schedule 
impact. Also, the technology requirements cannot be fully separated from 
consideration of what technology is available. Just as it does not make sense 
to require technology which will be beyond the state-of-the-art for 20 years, 
neither does it make sense to restrict the system to only that technology which 
has been previously developed and proven. An example would be a voice 
recognition system. Certainly the telepresence unit could function without it, 
but it can easily improve system performance. Since similar units are being 
flight tested onboard F -16 aircraft, and laboratory tests at JPL have verified 
the utility of voice controls for telepresence systems, it is reasonable to 
assume that reliable ground units will be available; a voice recognition 
command system is therefore part of the baseline for the initial telepresence 
system. 

1.3.2 NEAR TERM TECHNOLOGY REQUIREMENTS AND ASSESSMENTS 

1.3.2. 1 HUMAN FACTORS AND MAN/MACHINE INTERFACE 

1.3.2. 1.1 REQUIREMENTS 

From a human factors viewpoint, the more "transparent” the system, the 
better for operator performance. What this means is that the operator should 
not be performing tasks which would not have to be done If the operator were 
actually at the worksite. A mechanic rarely consciously considers how to 
position and move his arm while working: the control of his arm is autonomic, 
if he were forced to plan and think about each movement of his arm, his work 
would be slowed down considerably. A more extreme example is that of a runner. 
If a runner were to try to control his legs by actually thinking about the 
kinematics of motion, he probably would not be able to walk, let a lone run. 
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The goal of a transparent system is to maximize the natural control of the 
system by the operator. This should be done to the maximum amount possible 
without degrading other system operating considerations. For example, a system 
with maximum "transparency" would have anthropomorphic manipulators with the 
size and response time of human arms. This may not be desirable for a space 
system, because such arms may be too short to perform some desired tasks. 

Also, if the manipulators had the power to respond as quickly as a human arm, 
they would probably be too heavy for space use. 

The operator should also be provided with "natural" feedback and 
information whenever possible. Digital readouts provide useful information, 
but graphic or vector displays are often much easier to use. The operator 
should be able to gather the desired information without having to remove his 
attention from the task he is performing. This may lead to voice controls, 
"heads up" type displays, two operators per telepresence unit, etc. 

One useful way to think about this problem is to consider the design of an 
aircraft. Flight and auxiliary system controls are always considered as a 
whole, because they interact through both the plane and the pilot. For 
example, placing the landing gear lever next to the flap switch on an airplane 
is a poor design, even though it may be a convenient location. The same 
interaction of primary and auxiliary functions is present in the control 
station of a telepresence system. This consideration of human factors has been 
applied to the other technology requirements and assessments. 

1.3.2. 1.2 ASSESSMENT 

The nuclear industry has a large amount of experience with manipulator and 
remote workstation operations, but much of this experience has been with very 
limited manipulators and direct viewing of the worksite. A significant effort 
is now underway at Oak Ridge to analyze the remote operation problem from a 
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fundamental point of view, and to design teleoperator systems which can operate 
in and repair a permanently sealed nuclear facility. ^Auch of this data base is 
applicable to space telepresence, but caution must be used since the work 
environment and tasks are significantly different. 

Both government and industry have a large body of experience for aircraft 
and spacecraft cockpit design. Much of the data about displays, operator 
response time, and decision making capabilities should be very useful for 
designing the telepresence control and interface system. 

The biggest problem with the human factors area is that much work has been 
done, but it has been scattered between research centers, and the efforts have 
usually been somewhat piecemeal. For example, one center has investigated one 
type of controller and one type of arm, and another center has programmed a 
simulation and a vision system, but the work has never been put together. The 
Naval Ocean Systems Center in Hawaii (NOSC) has the most complete telepresence 
unit yet assembled, but the hardware is not up to date, and funding has limited 
the amount of studies which they have been able to perform. 

A research program needs to be started with simulation, experimental 
work, and hardware development proceeding together. This would permit valid 
experiments and tests of the man/machine interface. 

1 . 3 . 2. 2 VISION 

1 . 3 . 2. 2.1 REQUIREMENTS 

Vision is the most important of man's senses, and is particularly critical 
for space operations where audio and tactile cues are absent. The system 
described in this section is felt to be the desirable baseline for a 
telepresence system. 

The primary vision system should be a color stereo-optic device capable of 
being slaved to the operator's head position. The camera platform should have 
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at least 2 degrees-of-f reedom (DOF) (pitch and yaw), and the display should be 
visible from any head orientation. Remote cameras should be able to provide 
oblique and closeup views of the worksite. 

Stereo-optic vision has proven superior to monocular vision in performance 
tests, and provides the operator with depth perception capability. This 
capability will become important when guiding manipulator arms past obstacles. 
Color provides additional cues which aid in scene recognition and 
understanding. One of the best arguments for color vision is to view space 
operations (particularly in the orbiter payload bay) on a black and white 
display, and then on a color display. In many B&W images it is difficult to 
orient the image, let alone recognize components. The additional information 
provided by the color image usually solves problems of this type. Finally, 
because B&W vision will require use of TDRSS K band communications, but will 
only need a fraction of the K band capability, the inclusion of color vision 
should not be a major additional expense (see 1.3*2.6)> 

An operator reaching inside a spacecraft or working in a crowded 
environment would benefit from being able to pan and tilt the cameras by simply 
moving his head. This uses the natural motion of turning the head to change 
the scene being viewed. It also removes the need for always having to command 
camera position by voice or joystick control. To make this a workable system, 
a helmet mounted display is recommended. A binocular helmet mounted display 
would eliminate the need for complex optics usually associated with stereo 
vision, and allow greater comfort for the operator. Since the operator can 
select the initial point of operation, he can start work from a relaxed 
position, and then activate the helmet/camera position slave system. Two of 
the common complaints about stereo vision systems is that the optics cause 
headaches, and that the operator must hold his head in a fixed position. 

Helmet displays eliminate these problems. They allow the operator to choose 
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his own head position and, instead of using optics to create the stereo effect, 
they provide a separate image for each eye. 

Experiments with difficult lighting situations, limited access to the 
workspace, and operations requiring closeup views all suggest that additional 
cameras are desirable, and often necessary. Closeup views from the wrist of 
each manipulator arm and side views from either the body of the servicer or the 
elbow of the arm would appear to be best, but this determination would be part 
of an experimental program. An additional option is placing cameras on a 
multi-DOF platform such as a manipulator arm, or remotely controlled 
maneuvering unit, and allow the operator to move the camera platform around the 
workspace. 


1.3. 2 . 2. 2 ASSESSMENT 

NOSC Hawaii has developed a 3 DOF (roll, pitch, yaw) BSW stereo vision 
system which is helmet mounted and is slaved to the operator's head 
orientation. The components of this system are off the shelf technology from 
the 1970's. Small B&W CCD cameras have been developed by RCA, Fairchild, and 
Hitachi. The Fairchild camera has already been space qualified for use on 
pressure suit helmets during EVA. 

The primary vision system need not be CCD technology: space rated color 
cameras already in existence could do the job, but CCD cameras are smaller, 
lighter, use less power, and are are more reliable than the older vidicon types. 
Color CCD cameras are under development, and may become available at any time. 

Color display units are also under development. The Air Force has a high 
resolution, wide field of view system under development for flight simulators. 
The Visually Coupled Airbourne Systems Simulator (VCASS) demonstrates that 
helmet mounted color vision systems are within the state-of-the-art. Advances 
in light valves and fiberoptic bundles may eliminate the need for mounting the 
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CRT on the helmet, and allow less expensive large fixed position CRT's to be 
used. Alternately, commercial introduction of color, high resolution video 
screens using liquid crystal technology is expected by the middle of 1984* 

The hardware necessary for the vision system of a telepresence unit is 
nearly developed, and will require little NASA support. Work will be necessary 
to integrate and space qualify the system, but this should present little 
difficulty, as similar hardware has recently been qualified. The work that 
NASA will need to support is the testing of the vision system, and its command 
modes in conjuction with the rest of the telepresence system. 

Several options exist for commanding the vision system. Slaving the 
camera position to the operator's head orientation is useful for many tasks, 
but could be a disadvantage during long or repetitive tasks that do not require 
camera repositioning. Since there will probably be extra fixed displays for 
viewing by non-operator personnel, a fixed display should also be provided for 
the operator. This will facilitate discussion of problems and procedures with 
observers, and some operators may prefer to use it for certain tasks. 

Voice command of the vision system is a promising option. Speech 
recognition should become highly reliable in the next 2 to 3 years; it would 
free the operator's hands for manipulator control. 

Computer aided control is also a viable option. The camera can be locked 
onto the end effector or some portion of the worksite at the discretion of the 
operator. This can be accomplished with voice commands, by keyboard, or by a 
simple Joystick device. Also, sensors have been used in medical, and other, 
laboratory experiments which monitor the position of the operator's eyes to 
determine focal distance and target. An advanced vision system could monitor 
the operator's eyes to derive commands for the vision system. Such a system 
might be very natural and easy to use, but its feasiblity, usefulness, and 
human factors control laws must be determined by experiment before it can be 
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considered for use. 


Due to the different work regimes and tasks, the operator should be 
provided with several options, and allowed to determine which is preferable. 
Since there may be many different users and operators of a telepresence system, 
not Just highly trained "pilots", it should be flexible enough to adapt to the 
needs of various operators. 

1 . 3 . 2. 3 MANIPULATOR ARM 
1 . 3 . 2. 3.1 REQUIREMENTS 

Two 7 DOF manipulator arms should be used for a telepresence system. They 
are capable of reaching around objects, and provide dexterity similar to a 
human arm. If feasible, the arms should be anthropomorphic, because this 
requires a minimum of adaptation to the system by the operator. In some cases, 
anthropomorph i c arms may have difficulty performing certain tasks due to 
workspace and dexterity requi rements, so non-anthropomorph i c arms should be 
investigated. This is also discussed in section 2.5*7 • 

In addition to the main arms, there should be either a docking device or 
two grappling arms to lock the telepresence system in place relative to the 
worksite. A docking device is defined as a connection capable of transmitting 
reaction loads in all six axes; grapple arms (also known as anchor arms) are 
simple manipulators capable of grasping protrusions such as hand rails or 
structural members, and two would be required to provide a sufficiently rigid 
connection to the worksite. Without a docking device or grapple arms, 
thrusters would have to hold both the servicer and the spacecraft being 
repaired in position. This is a waste of fuel, and in most cases thruster 
performance would be inadequate. The docking device would be similar to the 
ones being planned for TMS orbital transfer operations. It is desirable to 
utilize the docking hardpoints for telepresence operations, because the 
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hardpoint will already be included in the design of any vehicle which can dock 
with TMS. Since the docking point on many spacecraft will be distant from the 
worksite, other means of attaching the servicer to the spacecraft being 
serviced are needed. Many of the spacecraft which are candidates for near term 
servicing will have RMS grapple fixtures and EVA handrails included in their 
design. One device could be used to grapple the RMS fixtures and lock the 
servicer in position, but this requires that several RMS fixtures be located at 
the proper positions to accomodate the needs of the servicer. Since many of 
the spacecraft are being designed without knowledge of the configuration and 
abilities of a telepresence servicer unit, this is a questionable option. A 
better choice is to have two simple arms with a generalized gripper or grippers 
which can use RMS grapple fixtures, EVA handholds, and structural members or 
other suitable hardpoints to lock the servicer In position. These arms need to 
be able to grapple in any orientation, so they will require 7 DOF; they do not 
require motors or means of locomotion, except for the grasping motion of the 
end effector. They can be positioned by the main arms, and use brakes or other 
devices to lock their position during the servicing operation. An alternative 
would be low power lightweight motors that are only used to position the arms, 
and again use brakes to lock their position during servicing activities. 

One of the key problems in manipulator design is compatibility with the 
spacecraft to be serviced. Short manipulator arms may not be able to reach a 
distant worksite; long arms may have trouble with flexibility, and may lack 
dexterity at a close worksite. The TMS/servicer spacecraft could crawl along 
the spacecraft to be serviced, or otherwise adjust its position, but the size 
of the TMS/servicer would .often preclude this technique. 

One solution is to use long, many DOF arms with "mini arms" that lock the 
manipulator to the spacecraft near the worksite. The control difficulties and 
Increased complexity of such a system make it an unlikely choice. Also, its 
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sized and weight could be prohibitive for small vehicles such as the TMS. It 
is possible to place a manipulator module on the end of the RMS. The module 
could be maneuvered into the desired position near the worksite and grapple 
arms could lock it into place. The long reach of the RMS and the high payload 
capacity of the shuttle make this a feasible option, but it would be limited 
by shuttle orbital and mission constraints (see section 2.4.1) • 

Another solution is a detachable servicer unit that leaves its TMS 
transporter and crawls along the spacecraft. This complicates the servicer 
design, and risks the loss of the servicer unit. 

Probably the best approach, particularly for near term operations, is 
either multiple size arms or a modular segment arm system (Figure 1.5)* The 
multiple arm approach selects from a set of different sized arms the proper 
size for a particular mission. This option would be best used with a 
telepresence unit based on the ground or at a space platform. This would 
allow the arms not being used for the present mission to be left behind, thus 
saving fuel and increasing payload capability. In some cases, more than one 
arm set would have to be carried, because of diverse servicing requirements 
for a single spacecraft. 

The modular arm system uses several different sizes for each 
shoulder-to-elbow and elbow-to-wr i st segment, etc. This system has the 
advantages of being able to tailor Itself for each mission, and of being 
capable of a limited degree of self-repair. It is also inherently more complex 
than the other options. 

In keeping with the NASA goal of an evolutionary system with capability 
increasing over time, there are some alternatives to traditional manipulator 
design that are worth investigating. Tendon activated manipulators have the 
inherent advantage of being able to place the motors closer to the "shoulder'* 
of the arm, and thus reduce the effects of inertia. This lighter arm has a 
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Figure 1.5: Manipulator .Arm Options 
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higher powei — to-weight ratio, and is capable of fast response similar to that 
of the human arm. Such arms also have more backlash and error in arm position 
and movements. This is not a problem because, unlike industrial robots which 
often move with open loop control, the human operator provides closed loop 
position and trajectory control, and can accomodate the arm ^s backlash. 

A much more advanced concept uses drive elements which are also structural 
elements to produce a truly flexible manipulator. An example is an arm 
constructed of stiff, but flexible wires which are held roughly parallel by 
widely separated nodes through which the wires pass. As tension is applied to 
a wire, the entire structure is deflected towards the wire under tension. 

This produces a manipulator flexible along it entire length. Such advanced 
arms are worthy of investigation, but the extremely large number of 
degrees-of-f reedom of such a system will require advanced computer control 
capabi 1 i t i es . 

1 . 3 . 2. 3. 2 ASSESSMENT 

The basic work necessary for a space qualified 7 DOF manipulator arm has 
been completed. The Proto Flight Manipulator Arm (PFMA) built by Martin 
Marietta for Marshall Space Flight Center is the product of this effort. The 
PFMA was designed so that the ground development and demonstration units could 
be modified for space use. The PFMA, or arms like it, could be used as 
manipulator arms for the initial near term telepresence system. 

A note should be made concerning the number of DOF of the manipulator arm; 
the 7 DOF In the manipulator discussed in this section is for the arm only, it 
does not include end effectors. End effectors can add several OOF, depending 
on their design. Also, some interchangeable end effectors (section 1.3. 2. 3) 
can incorporate some of the DOF normally associated with the manipulator arm 
into the end effector. For example, some interchangeable end effector designs 


1 . 3 . 13 



Include pitch, yaw, and roll motions often associated with the wrist of the 
manipulator . 

Docking fixtures and grapple points for the RMS are a well developed 
technology, and should not present any significant problems. Docking or 
grappling arms have not received much attention, and are a neglected 
technology. The basic technology for versatile docking arms Is Inherent In 
manipulator arm developments. An obvious solution to the problem Is to use a 
second set of manipulatpr arms as the docking arms. This Is an expensive 
option, and is probably not as desirable as designing dedicated docking arms. 
The dedicated arms could be designed without motors, or with very small ones 
just sufficient to lock the arms In place; the grapple arms could therefore be 
much lighter than the main manipulator arms. The grapple arms could be 

positioned by the main manipulator arms, and then locked in place by their own 

motors or brakes. A careful analysis of the size and shape of the 

telepresence unit and the worksite environments will be necessary prior to the 

design of the grapple arms, but the technology for such arms exists in present 
man i pul ators . 

The technology for multiple sized arms does not yet exist. Near term 
telepresence systems will probably have to work with single sized arms unless 
the development of modular arms is given a high priority. The difficulties 
with modular manipulator segments are not fundamental problems, but are 
composed of a myriad of engineering difficulties arising from attempting to 
provide automatic, lightweight interfaces that can transfer power, data, 
forces and torques, and capable of being interchanged in space. A less 
versatile, but more feasible option would be to have several different sized 
arms which are plug-in modules at the shoulder (Figure 1.6)* If the 
telepresence unit will be operating in a sortie mode from the ground or a 
space platform, it may not be necessary for the system to be able to change 
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its own arms, as it could be done by onsite humans. In either case, the 
development of interchangeable manipulators is feasible, but will have to 
overcome many engineering difficulties. 

Tendon actuated arms are certainly feasible, but probably could not be 
developed in time for use with a near term system. Flexible manipulators are 
presently only an interesting research topic. Both should be investigated, 
since more advanced telepresence systems could benefit from very dexterous 
h i gh-power-to-weight ratio manipulators. 

1.3.2.2| END EFFECTORS 
1 , 3 . 2. 4.1 REQUIREMENTS 

There are essentially two different approaches to the problem of end 
effectors; the hand or hand analogue, and the interchangeable end effector 
approach. From the viewpoint of providing the best telepresence (highest sense 
of presence at the worksite and capabilities nearest that of a human), the 
mechanical hand is probably best. From the viewpoint of feasibility and 
reliability, the interchangeable end effectors are probably best. 

As is quite obvious, human hands are remarkable constructions capable of 
almost any task. They are also very complex, which means any attempt to 
duplicate some or all of their capabilities will be a difficult undertaking. 

The biggest advantage of the hand type end effector is the operator's ability 
to perform dexterous operations with it. Two factors limit its usefulness for 
near term telepresence operations. First, most of the near term tasks do not 
require dexterity beyond the ability to grip an object or tool, so the need 
for the full dexterity of the hand is questionable. Second, the lack of force 
feedback would limit the usefulness of such an end effector. As discussed in 
section 1.3. 2 . 6 , the advantages of force feedback may be limited or eliminated 
by communications time delays. 
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Interchangeable end effectors are capable of performing the required near 
term telepresence tasks. In addition, a hand type end effector might be one of 
the end effectors carried by the telepresence unit. Advanced telepresence, 
particularly on a space station or in another short delay communications 
environment, will be able to take advantage of the additional dexterity and 
flexibility of a mechanical hand end effector. A general purpose gripping end 
effector or set of end effectors needs to be developed to manipulate various 
spacecraft components, and to allow the telepresence unit to attach itself to 
any structurally sound portion of the spacecraft it is servicing. 

1.3. 2. 4. 2 ASSESSMENT 

Interchangeable end effector designs have been developed, and laboratory 
demonstrations performed at MSEC and elsewhere. Some task-specific end 
effectors have been developed, and there should be no problems developing the 
additional ones necessary for actual telepresence use. A generalized gripping 
device has not yet been developed. Hands, prehensile tails, and tentacles may 
be the only true such devices, but enough work has been done to permit the 
development of a set of devices sufficiently capable of performing generalized 
gripping tasks. 

Dexterous end effector (band or band analogue) development has been 
sporadic at best, but recent work in prosthetics indicates that such devices 
are feasible. Sooner or later the limited dexterity of the non-hand end 
effectors will impact the performance or completion of a mission, and the need 
for a hand will become apparent. Since it Is difficult to construct a hand 
with motors at each joint, this would be a good candidate for a tendon 
actuation system. Such a development could be combined with a general tendon 
drive manipulator program. 

End effectors are a key component of a telepresence system, and have been 
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somewhat overlooked in past research. A good end effector will not necessarily 
make a good telepresence system, but a poor end effector guarantees a poor 
telepresence system. Near term telepresence systems should have no trouble 
being supplied with good end effectors, but advanced systems will require the 
development of more dexterous end effectors. 

1.3. 2. 5 SENSORS 
1.3. 2 . 5.1 REQUIREMENTS 

In addition to the vision system (section 1 . 3 * 2 . 2 ), various other sensing 
devices are required to provide information to the operator or the control 
system. Force and torque sensing ability is necessarily independent of whether 
force feedback to the operator is feasible. Commands from the operator may 
specify a certain level of force to be applied, and the telepresence unit will 
have to monitor its own force application, or structural and operational limits 
could be exceeded. It should be noted that there is a distinct difference 
between force sensing and tactile sensing. Tactile sensing is used to 
determine the texture of a surface, and can be used to sense slip (the sliding 
of an object or surface past an end effector) as well. Force sensing is used 
to observe and control the load being applied by an end effector, and is 
usually not able to differentiate between surfaces or identify slip. 

Tactile sensing is not required by near term telepresence activities, and 
this information is, at present, difficult to transfer to an operator. One 
exception to this is the special case of slip sensing. When applying large 
loads or performing tasks in which the end effector may lose traction with the 
object being manipulated, slip sensors can be very useful in preventing the 
end effector from losing Its grip on the object. The slip information can be 
sent directly to the manipulator control system, or to the operator via video, 
tactile, or audio signals. 


1 . 3.18 



Proximity sensors are very useful for grappling targets in an obscured 
vision environment. When reaching into an access hatch to disconnect an 
electrical connector, an audio or visual signal indicating distance to the 
connector (and/or closing velocity) can improve performance, and reduce the 
chances of a damaging collision. Even when vision is available, depth 
perception alone many not give sufficient or usable information, so the 
additional Information from the proximity sensor is desirable. 

Although a vision system decreases the importance of manipulator position 
sensors, they will be useful for a space telepresence system. Knowing the 
exact position of the arm in relation to the workspace permits the use of 
supervisory control systems as well as computer enhanced displays. For 
example, the range and rate of a manipulator closing on a target can be 
displayed to the operator as a visual or audio signal based upon the known 
position of the arm. Also, some control modes require either the absolute or 
relative position of the arm to be known in order for the control system to 
operate. 

The ability to determine the range and range rate, azimuth, etc. of a 
nearby target may not be necessary for telepresence, since the operator can 
obtain sufficient information from the vision system for manipulation tasks., 
but range and related information is necessary for approach and docking. Also, 
for work with large structures, such information would be useful. 

Force, torque, and position sensing capability are definite requirements 
for a telepresence system. Proximity and slip sensors are not mandatory, but 
are strongly recommended. Range and similar information is necessary for 
docking and may be useful for some telepresence tasks. 

1 . 3 . 2. 5. 2 ASSESSMENT 

Force and torque sensing is well within the state of the art. There are 
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two approaches which can easily be used. First, the load can be measured 
directly using strain gauges or a similar device on the manipulator structure; 
second, the current in the actuation motors can be measured to determine the 
load being applied. Both methods are simple, and can be easily implemented. 
For space use, fiberoptic strain and current sensors may be desirable because 
they are less sensitive to thermal distortion and electromagnetic interference 
(EMI). These sensors operate by changing their refractive index when under 
strain, and by altering the phase of a transmitted laser beam when placed In 
the field created by a current carrying wire. 

Position sensors are most commonly potentiometers attached to the 
manipulator arm joints, whose changing resistance indicates a change in joint 
angle. A more difficult, but more precise, method is to use optical encoders 
to measure the joint position and change of position. Both of these methods 
use sensors at each arm joint, and usually require knowledge of the past 
position history of the arm in order to determine arm position. A process 
called "Selspot" uses infrared LED ^s attached to the manipulator arm and 
infrared sensors placed nearby (such sensors could be placed near the camera 
platform) to determine the position of the arm relative to the sensors. This 
method can determine the arm position without knowledge of its position 
history, but It is sensitive to obstructions of the LED Infrared sources. All 
three technologies are within the state-of-the-art. 

Proximity sensors have been in use for various purposes for many years, 
and are a mature technology. JPL has recently developed and tested a 
proximity sensor for a grapple device to be used with the RMS. 

Tactile sensors have developed and tested at MIT and JPL, but would 
require significant development before any space use. Also, the problem of 
transmitting the information to the operator has not been solved. Current 
systems use visual displays of the tactile information, but direct tactile 
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communication with the operator would be more effective. Slip sensors are a 
less sophisticated version of tactile sensors, and are within present 
capabilities. An example of a simple slip sensor is a system of several 
microswitches built into the surface of a gripping device. As a rough surface 
or an edge passes a switch, a pulse is created by the opening or closing of the 
switch and communicated to the operator. The slip information does not present 
a display problem, because the operator only needs to know if slip is 
occurring, and as an option, the direction of slip. 

Accurate range information for docking and rendezvous can be obtained from 
radar similar to the Ku band system onboard the shuttle orbiters. The 
performance of these systems degrades at close range (less than 30 m) , and can 
damage delicate communications equipment. Laser radar can be extremely 
accurate at both long and close ranges, and can operate at low power levels and 
special frequencies to protect delicate equipment or nearby humans. The 
addition of retroref 1 ectors on the target offer accuracies beyond the 
capability of any other technology. JPL has developed a system for alignment 
of large space structures. The technology from this, and other experimental 
laser rangefinders and radar systems could easily be adapted for spacecraft 
docking purposes. 

1 . 3 . 2. 6 COMMUNICATIONS 
1 . 3 . 2. 6.1 REQUIREMENTS 

Although near term telepresence systems may be capable of some autonomous 
operations, most of the work will require direct human control. This means 
that a stable communications link must be maintained. This link must be of 
sufficient bandwidth to carry the commands to the spacecraft, and the video 
signal to the control station. The command link to the spacecraft requires 
only a low bandwidth signal, as video signals are not being sent to the 
spacecraft. The bandwidth will vary significantly depending upon how the 
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commands are sent. For example, a command to move a manipulator arm joint can 
be sent as a series of instructions (start, continue, stop) or as a single 
command (rotate X degrees). Regardless of the format used, the command uplink 
will be a small fraction of the downlink bandwidth used for video data. This 
raises the option of using a small bandwidth uplink and a high bandwidth 
downlink. Due to the desirability of stereo vision, a high data rate of at 
least 50 Megabi ts-per-second (Mbps) is desired. This provides the minimum 
resolution, scene size, and refresh rate necessary for effective B6W stereo 
vision. A higher bandwidth (200-300 Mbps) is highly desirable, since it 
provides higher resolution, larger scene size, faster refresh rates, and color 
capability. Also, auxiliary images may be simultaneously transmitted from 
remote cameras when the higher bandwidth is available. 

The communications system should be designed to minimize time delays and 
their effects on operator performance. A delay of approximately 0.5 seconds, 
due to the speed of light, Is unavoidable if a geosynchronous relay is to be 
used. Direct ground to spacecraft transmission has a smaller delay if the 
spacecraft is at a lower altitude than GEO, but ground station visibility 
decreases along with spacecraft altitiude, so this option has very limited 
usefulness. Space station to spacecraft communication also offers reduced time 
delays. These delays are negligible If the worksite is near the station, but 
as the distance to the worksite becomes greater, the problems of visibility 
limit the impact of this option. Since the 0.5 second delay Is unavoidable 
with the required geosynchronous relay, the operator will be in a “move and 
wait" control strategy. In this mode the operator makes a motion or gives a 
command, and then waits to observe the results before giving another command. 
Experiments with test systems indicates that performance continues to degrade 
with increasing time delays. 

The move and wait strategy does force the operator to carefully consider 
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his next move, but it also removes the ability for quick response and reaction 
to unexpected events. In addition, time delays render force feedback nearly 
useless, and may cause it to be disadvantageous. The feedback delay, coupled 
with the operator, could actually cause the system to become unstable. 

Predictive displays may help to reduce the effects of time delays (see 
section 2.5.6), but they are useless or harmful during unexpected events, 
because the operator may assume the display Is correct when it is actually 
wrong . 

In summary, high bandwidth communications (200 Mbps) are desirable, and 
approximately 50 Mbps is required for a viable black and white system. 

Time delays produce several undesirable effects. They slow down operations, 
limit fast response and ability to handle unexpected events, and render force 
feedback questionable at best. Every effort should be made to minimize the 
total time delay and its effects on the operator. 

1.3. 2 . 6. 2 ASSESSMENT 

The communications capability for a color stereo vision system presently 
exists. The TDRSS K band single access link can handle up to 3 OO Mbps. Some 
of the hardware for the spacecraft side of the link needs more work and space 
qualification, but there are no technological problems. S band hardware is 
cheaper, available for space use off-the-shelf, and has less stringent 
pointing requirements, but it cannot handle the 50 Mbps necessary for a high 
quality B&W system, much less color. Since the K band should be used for B£W 
vision anyway, adding the capability for color should not greatly increase the 
cost of the communications system. The time delay problem is complicated by 
the fact that very little data exists about the interaction of a human 
operator and a manipulator system with time delays. At present, time delays 
longer than 0.1 to 0.3 seconds significantly degrade performance. Performance 
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appears to decrease in a linear fashion with Increasing time delay. Figures 
1.7 and 1.8 show the time delays associated with the TDRS5 communications 
link. The delay on the downlink is 0.25 seconds from the spacecraft through 
TDRSS to White Sands and another 0.25 through a domestic satellite (DOMSAT) or 
ground lines. The delay in the uplink is 0.25 from White Sands through TDRSS 
to the spacecraft and another 0.25 through a DOMSAT or ground lines. The 
NASCOH block encoding can add up to 1.0 seconds to the uplink delay. The 
total delay could be reduced to 0.5 seconds if the control station were placed 
at White Sands. The alternative is very high speed land lines, and either a 
higher speed NASCOM block encoding scheme or abandonment of the NASCOM system. 
The simplest choice would be to place the control center at White Sands. The 
data may still require block formatting prior to transmission to TDRSS, but 
the delay should be significantly less than 1.0 second. Modern digital signal 
processing equipment should be certainly able to perform this function in less 
than 0.1 seconds. The shuttle has been suggested as a possible control 
station for a telepresence system. The only advantage of this option occurs 
when the worksite is close to the orb iter: in this case the time delay is 
negligible. When the worksite is not near the orbiter, as will be the case 
for many missions, the differing orbital constraints reduce visibility, and 
TDRSS must be used: any advantage to using the orbiter is then lost. Also, 
the orbiter would have to carry additional communications equipment, and 
mission operations would need to planned around the requirements of the 
communications system. A space station would provide an excellent base for 
telepresence operations. The telepresence system could be resupplied and 
maintained at the station, and could perform the same tasks for the station. 
The space station should have the capability for high bandwidth communication 
as part of its standard equipment. If the station is being used as a repair, 
servicing, and refurbishment site, many of the telepresence missions will be 
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Figure 1.7: TDRSS Communication Links 
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Figure 1.8: TDRSS Time Delays 



near the station, and will be able to operate without significant transmission 
time delays. The absence of time delays will permit the use of force feedback, 
which should improve the performance of delicate tasks. As the tasks it is 
performing take the telepresence servicer further from the space station, it 
will be necessary to use the TDRSS system, but this should present no problem. 
Also, near the time a space station is ready, a second generation of TDRSS type 
spacecraft should be available with enhanced capabilities. The space and 
ground segments of the required communications system are presently being 
deployed and tested. A K band link for the telepresence system needs space 
qualified hardware, but this should not present any problems. The time delays 
Imposed by the TDRSS/NASCOM system may seriously impact performance, and NASA 
should seriously consider placing the control station at White Sands. Before a 
decision is made, further laboratory work needs to be done to determine the 
impact of time delays on a telepresence system. A shuttle based control 
station is marginally desirable, but a control site based on a space station 
would be useful for work on or near the station, and would be able to use force 
feedback control because time delays would not be present. 

1 . 3 . 2. 7 CONTROL 
1.3. 2. 7.1 REQUIREMENTS 

Control technology is the key to telepresence. A properly designed 
control system will make up for many errors elsewhere, but an improperly 
designed control system will render the system near 1 y .usel ess , regardless of 
the quality of the other hardware. At the present time, much of the 
information necessary for this design task does not exist. The control 
technology requirements are presented as preliminary recommendations pending 
the results of the research suggested in the control technology assessment. 
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These technology requirements will, of necessity, be general in nature. 

The technology assessment will examine and recommend specific technologies. 

The primary function of the control system is to translate the commands of the 
operator into actions at the worksite. There are a number of techniques and 
control modes which can be used for this task, and they are discussed in the 
control assessment section. Research with telepresence simulators and ground 
development units will be necessary to identify the best control modes. 

The control system should also utilize technologies that reduce workload 
and improve performance. Several widely different display or command modes are 
presently under development, and will be discussed further in the control 
assessment section. A clear choice of display and command modes is not yet 
available. One probable exception is voice command systems. Since the 
operator usually uses his hands to control the manipulators, it may be 
difficult (or risky) to stop and throw switches to control displays and 
cameras, change control modes or end effectors, etc. In addition, after 
observing manipulators under human and computer control, an obvious conclusion 
is that the simple command "stop" may be the most valuable function of a voice 
control system. 

The telepresence system should also have a built in fail-safe capability. 
If communications are disrupted, the servicer should stop its actions. More 
advanced systems may take autonomous action, but this requires more 
intelligence than would be available for a near term system. A second type of 
fail-safe capability is also needed. An operator who sneezes while holding the 
manipulator controls, or accidentally gives improper commands, could cause 
significant damage. The ground or spacecraft computers must be able to 
recognize improper commands and refuse to obey them. For example, if the 
operator sneezes and suddenly jerks the manipulator controls during a delicate 
module exchange, the system should recognize that applying a large impulsive 
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force is Improper during this operation. 

One control task of critical importance is automated or remote docking. 
Since humans will not nominally be at the worksite, the docking of the 
telepresence system to the worksite is an important problem. Although remote 
docking is probably simpler than automated docking, the difficulties imposed 
by time delays, and the potential for the telepresence spacecraft to be 
"shadowed" by the target spacecraft, make automated docking extremely 
des i rabl e. 


1 . 3 . 2. 7 . 2 ASSESSMENT 

Of the several control techniques available for telepresence, only one has 
clearly shown itself to be highly desirable. Force feedback has consistently 
been found to improve operator performance during assembly and complex 
manipulation tasks. The more complex the task, the more force feedback helps 
the operator. Unfortunately, time delays longer than 0.1 to 0.3 seconds render 
force feedback useless or worse. Since these delays are an inherent part of 
any communications system using geosynchronous relays, force feedback to the 
operator may not be useful. This does not not mean that force control and 
measurement should not be used, but the operator may not be receiving direct 
feedback from the manipulators. A very promising option is to have a master 
control (either hand or exoskeletal controller) that measures the force applied 
to it, and commands the spacecraft to apply the same force. Figure 1.9 shows 
the difference between the two approaches. Part A shows force feedback with 
the operator closing the control loop, and part B shows force control with the 
spacecraft closing the control loop. 

Three basic control modes for a manipulator system are shown in Figure 
1.10. When in position mode, the manipulator matches the position of the 
controller. In rate mode, the manipulator matches the rate of motion of the 
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Figure 1,10: Basic Manipulator Control Modes. 



controller. Force control causes the force applied to the controller to be 
applied by the manipulator. If the manipulator is not working on a fixed 
object, then the manipulator will move at an acceleration proportional to the 
applied force. All of these options, or combinations of them, are viable for 
manipulator control, and arc within present technology. A clear choice of one 
system does not yet exist, and may not be desirable. Operators may prefer to 
have more than one control mode available for various tasks. JPL has developed 
a 6 DOF force reflecting hand controller, and also has a viscous damping model 
which helps operators deal with the differences in dynamics between master and 
slave arms. 

Much work has been done on the control of an individual arm, and tests of 
various controllers have been performed. Unfortunately, most of the tests 
mixed manipulator arms, controllers (master arms or hand controllers), control 
modes, and tasks. The data often is less than complete, and it is not possible 
to accurately compare the various techniques and hardware. Much time has been 
wasted because of funding difficulties in the past few years. What is needed 
is a simulation of the desired tasks, coupled with a system that can use 
various control modes to command the telepresence unit. The simulation should 
include time delays and a remote vision system. Thus, operator performance can 
be evaluated for each control mode, arm controller, and task. From this data, 
a system could be chosen for space use. Past research has provided the 
information necessary to start such a program, but accurate judgement must wait 
until the system can be tested as a whole. Just as the operational functions 
of a heads up display and its control modes for a fighter aircraft would not be 
selected until integrated testing was performed to determine the impact on the 
pilot and system as a whole, neither should the control modes of a telepresence 
system be selected prior to integrated testing. 

Several technologies capable of reducing the operator's workload and 


1 , 3-32 



I 


improving performance have been developed and tested. Speech recognition 
systems have reached the stage where they are ready for space use. Presently 
they are available for home computer systems, and are being tested in F-16 
aircraft. Such systems could switch control modes, control cameras, change end 
effectors, command data displays, provide emergency commands such as "stop", 
and initiate and terminate supervisory control algorithms. JPL has 
demonstrated the usefulness of voice command systems. 

Event driven displays have been developed by JPL. As a particular event 
occurs, the data being displayed changes to prepare the operator for the next 
task. For example, proximity data is displayed until a grapple has made 
contact with its target, then the display automatically switches to load and 
contact force data. A menu of tasks could also be shown. Similarly, control 
modes could automatically be changed by the occurence of key events. This type 
of technology does not require any knowledge or capability beyond what is 
available today, but experiments must be performed to determine where and when 
this technology should be applied. 

Supervisory control has been under extensive development at MIT, and 
elsewhere, for several years. This wor)c has been directed towards undersea 
vehicles and operations, but much of it is applicable to space use. Acoustic 
communications links (used for undersea applications) create time delays longer 
than those encountered in space. Supervisory control has proven itself capable 
of partially alleviating the effects of such delays, and in reducing operator 
fatigue during repetitive operations. Also, some tasks, such as removing a nut 
from a bolt, are better performed by a machine than a human. The human must 
repeat a complicated set of motions to unscrew a nut; a machine simply rotates 
the wrist until the nut has come loose. At the present state-of-the-art, an 
operator controls the manipulator to position it over the nut, and then 
commands the automatic nut removal; or, if the position of the nut relative to 
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the arm is known, then the machine can command the arm to move to the proper 
position and remove the nut. Supervisory control, at present, is similar to 
industrial robots in that it makes little use of feedback from its 
surroundings, and usually requires the objects in the workspace to be at 
predetermined positions during the system operation. It can be quickly 
reprogrammed by monitoring the operator's performance of a task, and then 
repeating the operator's actions. Proximity sensors can also be used to aid in 
locating a target, as can machine vision, but more research is necessary to 
make this a viable option. Supervisory control is not yet capable of following 
a command such as "remove the access panel". This requires the machine to 
determine a goal structure and implement a solution, and is beyond present 
technology. Supervisory control could be applicable to almost any telepresence 
task, and is the first step towards autonomous operations. 

Supervisory control is an attractive option for docking control. During 
the performance of a supervisory task, the spacecraft is essentially 
autonomous. However, the human operator can stop or modify the actions of the 
telepresence system, providing the advantages of both autonomy and human 
control. As for the actual technology involved, the US has extensive 
experience with piloted docking, but has yet to perform remote or autonomous 
docking. The problem is complex, but should not be difficult to solve; the 
Soviet Union has been performing remote and automated docking for several 
years. Their electronics technology is significantly less advanced than the 
US's, so duplicating or surpassing their capabilities is certainly feasible. 
Laser radar, rangefinders, and optical sensors are within present 
technological capabilities, and provide extremely accurate data for docking 
approach maneuvers. If the target spacecraft is equipped with laser 
retroref 1 ectors , laser systems can determine position and orientation to 
within microns. The most difficult part of developing an automated docking 
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system is the control algorithm and software design, particularly if grapple 
arms are used for docking (section 1.3*2. 3)* The determination of the 
location of suitable hardpoints for the grapple arms, and the guidance of the 
arms, will require a research effort with extensive validation testing to 
develop the necessary confidence in the system prior to deployment. 

As control technology and machine intelligence continue to advance, they 
will be integrated into the telepresence system with minimum impact to the 
system hardware. Most of the changes necessary will be in software, thus 
allowing the system to change in an evolutionary manner. 

1.3*2. 8 CONCLUSIONS 

Most of the component technologies for a near term telepresence system 
have been previously developed, or could be developed soon. The major tasks 
ahead are system integration, and control system selection and design. The 
control system selection and design requires the integration of a ground 
telepresence system to perform human factors research. Selection of specific 
component technologies can be done during this research and, as discussed in 
section 1.5, a demonstration system flown by 1992. 

1 . 3.3 ADVANCED TECHNOLOGY (LONG TERM TELEPRESENCE TECHNOLOGY) 

The long term (post 1995) telepresence system will be able to take 
advantage of the advances in artificial intelligence. Advances in manipulator, 
sensor, and other technologies will have important effects, but the key to the 
system will be intelligent information processing and decision making. 

Some of the technologies discussed in this section may be available prior 
to 1995» but many will require years of development, and may not be available 
until post 2000. The volatility and rapid expansion of computer and machine 
intelligence technology render forecasts in this area questionable. 
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The far term telepresence system will have two different modes of 
operation; full telepresence and advanced supervisory control. The remainder 
of this section will focus on the advanced technologies, beginning with full 
telepresence and control technology. 

1.3.3. 1 full telepresence 

At this level, the operator actually feels as i f he were at the worksite 
and performs naturally, taking advantage of experience, learned reactions, 
expertise, and human decision making abilites. This type of system should not 
require training beyond a simple introduction to the system, because it will 
operate in a manner similar to the human. The manipulator arms may not be 
anthropomorphic, but the system will accept and adapt anthropomorphic input. 

The system will have the capability to interact with the operator in natural 
language. An advanced "user friendly" telepresence system is not significantly 
more difficult to construct than one which is not user friendly. All of the 
developments necessary either make the system more effective (easy to use 
manipulators) or will be developed for other purposes, and could easily be 
incorporated into the system (natural language interfaces). 

Some problems will still exist despite any advances. Time delays will 
always exist, as long as the worksite is a long distance from the control 
center. Predictive displays and possiblities such as predictive force feedback 
can reduce the effects of time delays, but not completely eliminate them. 

1.3. 3 . 2 CONTROL 

1.3. 3. 2.1 SUPERVISORY CONTROL 

The utilization of supervisory control technology does not have to wait 
until post 1995. but the more advanced forms discussed here will require 
advances in machine vision and artificial intelligence. Present supervisory 
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control systems operate similarly to industrial robots. They cannot respond to 
changes in the environment, or to anomalous situations. More advanced 
supervisory systems will respond to higher level instructions, and will have 
the capability to perform complex tasks and make its own decisions. For 
example, it might understand and implement the instruction "replace amplifiers 
6 and 7"* It would look up the position of the parts, open the access panel, 
remove the module, replace the amplifiers, and return the module to its proper 
position. At this point the difference between autonomous operation and 
supervisory control becomes blurred. Thus, advanced supervisory control will 
be a natural step on the path to autonomous operations. An example of a 
mid-level supervisory control system in operation is presented in Figure 1.11. 

A remote human indicates an access panel to be removed. The onboard computer 
recognizes the type of panel, schedules the tasks needed to remove it, picks 
the first screw, determines that it is a ph i 1 1 i ps head, selects the correct 
tool and removes the screw, stores it if necessary, and goes on to the next 
task in the sequence. 

A telepresence system with advanced supervisory control has several 
desirable features. It is very useful for tasks which are severely impacted by 
the effects of transmission time delays. Such a system would rely on limited 
machine intelligence to deal with departures from nominal procedure. Since it 
would perform many tasks semi -autonomous 1 y , it would have reduced dependence on 
communications links and ground commands. Extra capabilities not found in 
human operators, such as i nf i nte roll wrists, extreme patience, etc.) are 
easily incorporated in the system software. Tasks which are boring, fatiguing, 
repetitive, or otherwise distasteful to human operators can be performed by the 
supervisory control system. 

All levels of supervisory control can be developed in parallel with the 
telepresence system. The supervisory system is implemented in software, and 
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can be added to a telepresence unit with minimum impact on the hardware. 
Particularly advanced control modes may require upgrading of the onboard 
computers, but should not affect the rest of the system. 

1.3. 3. 2. 2 ADAPTIVE CONTROL 

Adaptive control is desirable, because it increases the flexibility and 
capability of the telepresence system. As telepresence operations become more 
complex, additional flexibility will become crucial for mission performance. 

Near term missions require limited dexterity, plus the capability to 
perform contingency operations. As confidence in telepresence and our 
technological capability grows, so will the demands placed upon the system. 
Also, the increasing number and importance of spacecraft will create an 
increased need for reliable contingency operations. 

Adaptive control, in the most advanced case, is an artificial intelligence 
technology that measures and observes the performance of a given task or set of 
tasks. If the performance is not as desired, the system will attempt to 
improve It. This may be done via an analysis of the problem to find a 
solution. If this is unsuccessful, the computer may make small test changes in 
procedures to try to find a solution via experiment. A less sophisticated 
example of this can be implemented mathematically in control algorithms using 
techniques such as hill climbing. 

Adaptation occurs in most tasks which humans perform. For example, in a 
typical manipulatory task, many individual adaptations take place. If a screw 
being removed from a panel does not turn as easily as it should, the operator 
tries changing the angle of the tool or. increasing the torque. Note that the 
angle and torque are not suddenly increased to preset values, but are gradually 
increased until the desired results are achieved. Also, the angle and torque 
are not modified by extreme amounts, thus preventing damage to the equipment. 
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Adaptations of this type occur often during assembly or repair operations. 
A control system capable of adaptation would aid operator control, and improve 
the reliability of supervisory and autonomous control. Much theoretical and 
experimental work remains to be done in this area, from algorithm and software 
development, through studying the interaction of human operators with an 
adaptive control system. 

1.3. 3 . 2. 3 SUMMARY 

Once advanced supervisory and adaptive control systems are developed, 
some fundamental questions will need to be answered by experiment and 
simulation. The appropriate levels of supervisory control can be determined, 
so tasks can be assigned to man or machine based upon performance. Adaptive 
control strategies can be identified which allow deviation from planned 
procedure to improve performance, but not beyond safe limits. Interaction 
between man and machine, and the results of past human factors experience, can 
be studied to begin identifying the <^timum mix of man and machine control. 

1.3. 3 . 3 INTELLIGENT VISION 

Advances in image processing and understanding will radically alter 
machine understanding and interaction with its environment. Computer control 
systems will be able to monitor all facets of the work environment, instead of 
having to rely on a few limited sensors, as is presently the case. The direct 
impact on telepresence will appear in high reliability predictive displays, 
computer enhanced images, and supervisory control. Enhanced images might 
contain hidden line details, suggested manipulator paths, caution indicators, 
and other useful data. This should reduce the impact and risks imposed by 
transmission time delays. 

As machine image recognition allows the computer an improved understanding 
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of its environment, it increases the capacity for autonomous or semi -autonomous 
operations. Supervisory and adaptive control will be significantly improved by 
computer vision technology. Once this type of system is developed. It will 
form the basis for automatic hand-eye coordination manipulators. Such a system 
could, eventually, be capable of nearly all the control tasks associated with 
the human operator. 

True machine image recognition and understanding may be a post-2000 
technology, but many of the advantages of such a system can be achieved by 
intelligent use of available technology. For example, laser scanners could be 
used to read a bar code or similar device attached to spacecraft and 
manipulator components. Objects could be thus identified, without requiring 
the computer to truly recognize the target. Computers could then enhance the 
image or perform control tasks, without performing difficult and complex image 
recognition functions. An example of this particular technology has been used 
in supermarket checkout stands for several years. 

If the image processing were performed onboard the telepresence unit, 
computer control of important tasks could be performed without the difficulties 
of transmission time delays. A side benefit is that the video image sent to 
the ground could use a variety of techniques to greatly reduce the required 
transmission bandwidth. 

1.3.3.i+ MANIPULATORS AND END EFFECTORS 

Manipulators and end effectors should both benefit from improved motors 
and structural materials. Advances in tendon actuated devices should allow 
arms with fast responses and high ratios of power to arm-weight, because the 
drive motors could be mounted in the spacecraft near the shoulder. Other 
possibilities include mechanical "muscles", which would permit very dexterous 
and powerful manipulators. 
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The development of dexterous end effectors should be the most important 
development in this area. This will permit the performance of very intricate 
operations, which previously could only be performed on the ground or in a 
pressurized environment. The development of pressure suits with improved 
dexterity could also allow performance of dexterous operations in vacuum. 

1 . 3 . 3. 5 SENSORS 

This discussion of sensors will also Include visual sensors, because the 
intelligent vision section focuses on machine image processing. 

Optical sensors should exceed human vision in resolution and field of 
view. CCD technology will permit high resolution sensors to be placed 
virtually anywhere desired. Advanced control systems will require the 
determination of vision sensor requirements based on the needs of machine 
vision, rather than human vision needs. 

Laser imaging and scanning systems may be highly useful for providing 
additional information to computer control systems, but the data they produce 
will be useless to a human operator. 3D or holographic images are an 
interesting alternative to stereo vision systems, but do not at present seem 
to offer significant advantages over a stereo vision system. 

Genera] advances in force and proximity sensors can be expected, but their 
impact will not be very important. Acoustic sensors, which can be attached 
directly to a structure to test for failures, will be readily available. 

The development of a reliable tactile information transfer system would 
be very desirable, if the operator is not using a system with significant time 
delays. The utility of tactile information is degraded by time delays, as is 
force feedback data. Tactile sensors are being developed today with success, 
but little work has been done on direct information transfer to the operator. 
Tactile data is presently transferred to the operator as a visual signal, 
instead of a tactile response. 
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1.3. 3. 6 COMMUNICATIONS 


Communications technology is advancing rapidly, allowing increased 
bandwidth and data compression capability. Optical communication technology 
offers data rates and bandwidths far beyond anything in use today. 

Transmission of very high resolution, wide field of view images will be well 
within communications system capabilities. Advances in computer image 
processing will reduce the need for this increased capability, because the 
processed images could require less bandwidth than present data links. For 
example, a black region in an image requires several bits for each pixel 
(Depending on intensity ranges, color selection, etc.). The same image 
requires far less data to be transmitted if the image is processed. The 
computer can define the edges of the region and send instructions to the ground 
system to fill in the region, thus reducing the number of bits per pixel to a 
fraction of the unprocessed transmission. 

Advances in signal processing, encoding, and optical fiber land line 
networks will permit placement of telepresence control centers at any location 
without concern for ground time delays. Such delays will continue to exist, 
but will be due almost entirely to signal travel delays, and thus be the 
minimum delay possible. 

The lengthy delays imposed by geosynchronous relays may be eliminated by 
transmitting to neighboring satellites, and making use of ground based optical 
network links. As space operations expand and the number of spacecraft 
increase, the opportunity for using multiple spacecraft and non-geosynchronous 
relays will increase. 

l.A FACILITIES ASSESSMENT 

The facilities assessment is a compilation of available government. 
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Industrial, and academic centers capable of contributing to telepresence 
development. Primary attention was focused on facilities with expertise in 


telepresence and space operations. Other fac i 1 i t i es with capabilities in 
marine and nuclear remote operations, biomechanical engineering, and automation 
and robotic assembly are also included. Six centers with expertise in 
telepresence technology and space operations are discussed at the beginning of 
this section. Other facilities and their abilities are listed at the end of 
this section, followed by a brief summary of the facilities assessment. 


l.lt.l NASA MARSHALL SPACE FLIGHT CENTER (MSFC) 


50 FT X 90 FT AIR-BEARING TEST FACILITY 

- TESTS OF TELEOPERATED MANEUVERING SYSTEM (TMS) 

- DOCKING MECHANISMS 

- ADVANCED MANIPULATOR SYSTEMS 

- DEPLOYABLE STRUCTURES 

- MOBILITY UNIT/DOCKING TEST FACILITY 

- EXTENDABLE DOCKING PROBE 

- 5 DOF MANEUVERING UNIT 

- MOBILITY UNIT CONTRCTL PANEL 

ORBITAL DOCKING SIMULATION 

- AUTOMATIC RENDEZVOUS AND DOCKING 

ORBITAL CONTACT DYNAMICS 

MOVING-BASE SIMULATOR USING SIX DOF MOTION SYSTEM 

ELECTRONICS AND CONTROL LABORATORY 

- PROTOFLIGHT MANIPULATOR ARM 

- OPERATOR CONTROL STATION 

- TELEMETRY STATION 

- TELEOPERATOR CONTROL STATION DEVELOPMENT 

- RANCHO ANTHROPOMORPHIC MANIPULATOR 

- ISOMETRIC HAND CONTROLLER 

- EXTENDABLE STIFF-ARM MANIPULATOR 

- COMPUTER IMAGE AUGMENTATION AND ENHANCEMENT 

- STEREO VISION SYSTEMS 

- VISION SENSORS 

INTEGRATED ORBITAL SERVICER ENGINEERING TEST UNIT 

NEUTRAL BOUYANCY TANK FOR EVA AND SPACE OPERATIONS SIMULATION 


MSFC was one of the early centers conducting research in remote space 
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operations, and was heavily involved in the TRS program to reboost Sky lab to a 
higher orbit. The facilities for docking and contact dynamics simulation are 
excellent. A large air bearing test facility is nearing completion, and a 6 
DOF moving base simulator is operational. The Integrated Orbital Servicer 
Engineering Test Unit is used to simulate module exchange by either onboard or 
remote computer control, and can be controlled directly by human operators. 

The Proto Flight Manipulator Arm (PFMA) is the only space specific manipulator 
currently available. Built by Martin Marietta, it was specifically designed 
for space use, and requires only minor modifications for space qualification. 

It provides an ability to use a space specific manipulator arm in ground 
simulations. Some of the facilities at MSFC, as with others around the 
country, have suffered from a lack of funding following the cancellation of the 
TRS program. Renewed interest in teleoperation and automation has begun to 
reverse this trend. 


1.1+.2 NASA JET PROPULSION LABORATORY (JPL) 


ADVANCED TELEOPERATOR DEVELOPMENT LABORATORY 

- DEVELOPMENT OF FLEXIBLE SENSOR-AIDED AND COMPUTER-AIDED 

MANIPULATOR CONTROLS 

- MECHANICAL HANDS WITH SMART SENSORS 

- EFFICIENT MAN-MACHINE INTERFACES 

- PROXIMITY, FORCE-TORQUE, TOUCH AND SLIP SENSOR DEVELOPMENT 

- EVENT-DRIVEN DISPLAYS 

- VOICE RECOGNITION TO CONTROL DISPLAY AND MANIPULATOR MOTION 

- SENSOR AUGMENTATION OF SHUTTLE RMS 

ROBOTICS DEVELOPMENT LABORATORY 

- MACHINE INTELLIGENCE 

- FAULT TOLERANCE 

- ROBOTICS TECHNOLOGY DEVELOPMENT 

- COMPUTER VISION 


JPL has been working for many years on manipulator control, sensors, and 
command/operator interface research. JPL has the most advanced manipulator 
control systems currently in operation, including proximity, force, and tactile 
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sensors, voice command, force feedback controllers, and event driven displays 
for command input. 

This expertise in control systems is augmented by experience in machine 
intelligence, vision, and fault tolerance. 

As with nSFC, research at JPL was impacted by the decline of interest in 
teleoperation during the late 1970 ^s. The renewed interest in all aspects of 
automation technology should reverse the adverse effects of the late 70 's. 


l.lt.3 NASA LANGLEY RESEARCH CENTER 


INTELLIGENT SYSTEMS RESEARCH LABORATORY 

- 7 LSI - 11/03 COMPUTER UNDER PARALLEL ASYNCHRONOUS CONTROL 

- 2 UN I MATE 600, 6 DOF MANIPULATORS WITH FORCE/TORQUE TRANSDUCERS 

- END EFFECTOR RESEARCH 

- EXOSKELETAL SUIT FOR TELEOPERATION SYSTEM 

- REAL TIME SIMULATION OF A LINEAR 5 DOF MANIPULATOR (NONLINEAR 

DYNAMIC MODEL FOR A GENERAL MANIPULATOR IS UNDER DEVELOPMENT) 

- VAX 11/750 WITH LISP CAPABILITY 

- ACTIVE AND PASSIVE COMPLIANCE CONTROL STUDIES 

- HONEYWELL STEREO TV VIEWING SYSTEM - ON ORDER 

- RECONFl CURABLE TELEOPERATOR/ROBOT 1 C CONTROL STATION - UNDER 

CONSTRUCTION 

- INVESTIGATION OF TELEOPERATION TIME DELAY EFFECTS 

- PRELIMINARY GROUND CONTROL STATION DESIGN AND REMOTE ORBITAL 

SERVICING SYSTEM (ROSS) DEVELOPED BY MART I N -MAR I ETTA UNDER 
LRC CONTRACT 

- ARTIFICIAL INTELLIGENCE APPLICATIONS TO AUTOMATED DECISION 

MAKING DEVELOPED BY MARTIN-MARIETTA UNDER LRC CONTRACT 


NASA Langley is becomming very active in the automation and telepresence 
research efforts. Recent acquisition of modern computers and industrial robot 
arms provides the capability for robotics experimentation. Construction of a 
reconf i gurabl e teleoperation control station, and acquisition of a stereo 
vision system, will provide a simulation and test capability for telepresence 
control concepts. Additional contractor support through programs such as the 
Remote Orbital Servicing System (ROSS) provide Langley with a broad base for 
telepresence research and development. 
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1.4.4 MARTIN MARIETTA AEROSPACE 


SPACE OPERATION SIMULATIONS LABORATORY 

- TELEOPERATOR MANEUVERING SYSTEM (TMS) SIMULATION 

- 6 DOF MOVING BASE CARRIAGE FOR RENDEZVOUS AND 

DOCKING SIMULATIONS 

“ SHUTTLE CARGO BAY AND AFT FLIGHT DECK MOCKUPS 

- NEUTRAL BUOYANCY TANK FOR EVA AND IVA SIMULATIONS 

- LARGE SCREEN DISPLAY 

- MANIPULATOR DESIGN EVALUATION 

MANIPULATOR DEVELOPMENT LABORATORY 

- TESTING OF REMOTE CONTROLLED MANIPULATOR SYSTEMS 

- QUARTER-SCALE MANIPULATOR ARM CAPABLE OF SIMULATING THE SHUTTLE 

RMS, INCLUDING STEREO TV, FORCE FEEDBACK, BACK DRIVABILITY, 
AND POSITION CONTROL 

MAN-COMPUTER INTERACTION LABORATORY 

- VOICE RECOGNITION/SYNTHESIS 

- HIGH-RESOLUTION COLOR-RASTER GRAPHICS DISPLAY SYSTEM 

- MAN-COMPUTER INTERFACE DESIGN 

- REMOTE MANNED-CONTROL OF FREE FLYING SPACECRAFT 
SIMULATION SUPPORT LABORATORY 

REMOTE ORBITAL SERVICING SYSTEM CONCEPT DEVELOPMENT 
FOR NASA LRC 

MACHINE INTELLIGENCE UNIT 

- EXPERT SYSTEMS 

- NATURAL LANGUAGE DEVELOPMENT 

- KNOWLEDGE REPRESENTATION 


Martin Marietta has been involved in automation for several years, and was 
heavily involved in the TRS program. Simulation capabilities include shuttle 
mock ups, computer simulations, 6 DOF moving base simulator, and EVA 
simulations. Background experience includes Skylab docking and EVA, TMS 
studies, and MMU simulations. Present hardware research includes the PFMA 
development for MSFC, artificial intelligence, man/machine interface design, 

RMS quarter scale simulator with stereo display and force feedback, and the 
ROSS study for NASA Langley. Unlike other research and development centers, 
Martin Marietta appears to have been able to maintain and even improve its 
facilities, despite the decline in NASA funding during the late 70's. 
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GRUMMAN AEROSPACE CORPORATION 


GRUMMAN IS INVOLVED IN EXPLORING THE USE OF 
MANIPULATORS FOR THE SERVICING OF SATELLITES 
IN SPACE 

GRUMMAN IS USING A BILATERAL FORCE REFLECTION 
LABORATORY MANIPULATOR TO STUDY TELEPRESENCE 
TECHNOLOGY ISSUES RELATED TO SPACE OPERATIONS 

THE GRUMMAN LARGE AMPLITUDE SPACE SIMULATOR 
(LASS) TEST FACILITY IS USED TO SIMULATE MISSION 
APPLICATIONS, DEVELOP SPACECRAFT SYSTEM REQUIRE- 
MENTS AND INTERFACES, AND EVOLVE TELEPRESENCE 
FLIGHT SYSTEM DESIGNS 

GRUMMAN HAS ALSO DEVELOPED A REMOTE MAINTENANCE 
SYSTEM FOR THE STUDY OF NUCLEAR VESSEL SERVICING 


Grumman is a recent entry into the space teleoperation field, but is 
advancing rapidly. A teleoperation laboratory using manipulators developed for 
the nuclear industry is in operation. Remote vision systems are under 

development. Tests have included a spacecraft servicing (via teleoperation) 

» 

simulation in which a thermal blanket is removed, an access panel is opened, 
electrical and mechanical connections operated, and a module removed and 
reinserted Into the simulated spacecraft. Also available is a 6 DOF moving 
base aircraft simulator, which has been used for teleoperation tests. It is 
important to note that this work has been supported by I R&D funds. 


1.4.6 MASSACHUSETTS INSTITUTE OF TECHNOLOGY (MIT) 


SPACE SYSTEMS LABORATORY 

- NEUTRAL BUOYANCY SIMULATION OF EVA 

- LABORATORY INVESTIGATION OF HUMAN FACTORS AND CONTROL SYSTEM 

DESIGN FOR TELEOPERATION 

- BEAM ASSEMBLY TELEOPERATOR (BAT) FOR NEUTRAL BUOYANCY 

SIMULATION OF TELEPRESENCE 

- ARAMIS PHASE II (TELEPRESENCE) STUDY FOR NASA MSFC 
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ARTIFICIAL INTELLIGENCE LABORATORY 

- DESIGN OF DEXTEROUS (TENDON-ACTUATED) HANDS 

- MANIPULATOR DESIGN 

- HIGH RESOLUTION TACTILE SENSORS 

- STRAIN GAGE FORCE SENSORS 

- DEVELOPMENT OF HIGH LEVEL COMPUTER LANGUAGE BASED ON A 

GEOMETRIC MODELING SYSTEM 

- ROBOT VISION AND PERCEPTION 

- FUNDAMENTAL RESEARCH INTO ARTIFICIAL INTELLIGENCE CONCEPTS 

MAN-MACHINE SYSTEMS LABORATORY 

- SUPERVISORY CONTROL OF UNDERSEA TELEOPERATORS 

- TACTILE SENSING FOR OCEANBOTTOM OPERATIONS 

- VISION STUDY EVALUATING TRADEOFFS OF FRAME RATES. RESOLUTION, 

AND NUMBER OF GRAY SCALE LEVELS 

- OBJECT SHAPE DETERMINATION FROM TACTILE SENSING 

- PROSTHETICS RESEARCH 


The Space Systems Laboratory (SSL) provides MIT with expertise in space 
operations, human factors, EVA, and neutral buoyancy simulation. MIT has 
acquired more neutral buoyancy space structure assembly time than any other 
organization. The development of the Beam Assembly Teleoperator (BAT) is 
designed for neutral buoyancy simulation of telepresence, and to provide data 
to supplement the EVA simulations (Figure 1 . 12 ). The SSL first became involved 
in telepresence in 1978 while under contract to MSFC to study “Extraterrestrial 
Processing and Manufacturing of Large Space Systems". A telepresence servicer 
unit was conceptualized as a part of that study (also known as the Free-Flying 
Hybrid Teleoperator) and is shown in Figure 1 . 13 * 

The Artificial Intelligence Laboratory (Al Lab) has been at the forefront 
of computer and automation technology since Its formation. The term 
"Telepresence" was invented by the Al Lab ^s founder. Prof. Marvin Minsky. 
Besides A I research the A I Lab is involved in machine vision and perception, 
tactile sensor development, and the design of dexterous manipulators and end 
effectors. 

The Man-Machine Systems Laboratory of the Department of Mechanical 
Engineering has pioneered quantitative human factors research and undersea 
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Figure 1.13; Conceptual Telepresence Servicer Unit 
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teleoperation development. Present laboratory projects include vision systems 
evaluation, prosthetics development, and object shape determination via tactile 
sensors. The development of supervisory control systems such as Superman and 
MMIT has been one of the laboratory's most significant contributions to the 
teleoperation field. This research is continuing, and more advanced research 
systems are under development. 


l.it.7 OTHER TELEPRESENCE RELATED FACILITIES WITH SPACE EXPERIENCE 


ESSEX CORPORATION 

- LARGE SPACE SYSTEMS MAN/MACHINE ASSEMBLY ANALYSIS 

- TELEOPERATOR AND ROBOTIC SYSTEM RESEARCH AND DEVELOPMENT 

- MANIPULATOR, VISION, AND MOBILITY SYSTEM RESEACH 

- EVA MOCKUPS 

- HUMAN FACTORS FOR REMOTE SYSTEMS 

LOCKHEED MISSILES AND SPACE 

- SATELLITE SERVICING STUDY FOR NASA-JSC 

- EVA SERVICING STUDIES FOR SPACE TELESCOPE 

• 

SPAR AEROSPACE LIMITED - CANADA 

- SHUTTLE REMOTE MANIPULATOR SYSTEM (RMS) 

- RMS CONTROLS AND DISPLAYS 

TELEOPERATOR SYSTEMS CORPORATION 

- LABORATORY MANIPULATOR SYSTEM FOR GRUMMAN AEROSPACE ^S 

DEXTEROUS MANIPULATOR TEST PROGRAM 

- REMOTE HANDLING SYSTEMS FOR MAINTENANCE OF FUSION REACTORS AND 

PARTICLE ACCELERATORS 

VOUGHT 

- TELEOPERATOR MANEUVERING SYSTEM (TMS) 


These corporations are either involved in hardware design and test for 
remote space operations, or are studying space teleoperation. 

Essex corporation has experience in EVA neutral buoyancy simulation 
support, large space structures assembly analysis, and hardware design and 
support for air bearing floor docking and manipulation simulations. 

Teleoperator Systems Corporation is not explicitly concerned with space 
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operations, but their iong experience with manipulator design and construct ion, 
coupled with their work for Grumman Aerospace, makes them a valuable resource 
for manipulator design. 


I.4.8 OTHER TELEPRESENCE RESEARCH FACILITIES 


NAVAL OCEAN SYSTEMS CENTER (NOSC) - HAWAII 

- UNDERSEAS TELEPRESENCE 

- STEREO-OPTIC VISION 

NAVAL OCEAN SYSTEMS CENTER (NOSC) - SAN DIEGO 

- UNDERSEAS TELEPRESENCE 

UNIVERSITY OF FLORIDA, CENTER FOR INTELLIGENT MACHINES AND 
ROBOTICS 

- DESIGN AND ANALYSIS OF MANIPULATOR ARMS 

- ACTUATORS, SENSORS AND VISION 

- MAN-MACHINE INTERFACE 

- NUCLEAR TELEPRESENCE 

UCLA, SCHOOL OF ENGINEERING AND APPLIED SCIENCES 

- TELEPRESENCE STUDY FOR NOSC 

UNIVERSITY OF ARIZONA 

- TELEOPERATOR STUDY 

SRI INTERNATIONAL 

- 1977 TELEPRESENCE STUDY FOR NASA-AMES 

- MACHINE VISION, PERCEPTION, AND INSPECTION 

PERCEPTRONICS 

- TELEOPERATION INVESTIGATION EXPERIENCE (NOT PRESENTLY BEING 

PURSUED) 

OAK RIDGE NATIONAL LABORATORY 

- SPEECH CONTROL 

- NUCLEAR TELEPRESENCE 

- RADIATION-PROOF MOTORS, LUBRICATION AND HIGH TEMPERATURE 

OPERATION 

DEPARTMENT OF ENERGY (DOE) 

- REMOTE HANDLING OF NUCLEAR WASTE DISPOSAL 

FERMI NATIONAL ACCELERATOR LABORATORY 

- NUCLEAR TELEPRESENCE 
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OTHER APPLICABLE FACILITIES 


CHARLES STARK DRAPER LAB, INC. 

- ROBOTIC ASSEMBLY 

STANFORD UNIVERSITY 

- SENSORS. MANIPULATORS, HANDS. VISION, ROBOTIC CONTROL 

CARNEG I E-MELLON UNIVERSITY, ROBOTICS INSTITUTE 

- MANIPULATION, PERCEPTION, SENSORS (TOUCH, FORCE, PRESSURE, 

TEMPERATURE, VIBRATION AND CHEMICAL), LEGGED MOTION ANALYSIS, 
MANUFACTURING, ARTIFICIAL INTELLIGENCE 

RHODE ISLAND UNIVERSITY 

- DEXTEROUS END EFFECTORS, ARTICULATED HANDS, PERCEPTION, TOUCH 

SENSING, ROBOTIC MANUFACTURING 

GENERAL ELECTRIC 

- ROBOTIC ASSEMBLY, SENSOR INTEGRATION, INTERACTIVE DISPLAYS 
AUTOMAT IX 

- COMPUTER VISION, ROBOTICS 
UN I MAT I ON 

- MAJOR INDUSTRIAL ROBOT MANUFACTURER 

MACHINE INTELLIGENCE CORPORATION 

- COMPUTER VISION, ROBOTICS 

CINCINNATI MILACRON 

- MAJOR INDUSTRIAL ROBOT MANUFACTURER 

NATIONAL BUREAU OF STANDARDS 

- SENSORS, CONTROL, VISION, AUTOMATED MANUFACTURING 

VETERANS ADMINISTRATION, REHABILITATION ENGINEERING CENTER, 

STANFORD UNIVERSITY. 

- BIOMECHANICAL ENGINEERING, HUMAN-MACHINE INTERFACE 
UNITED STATES AIR FORCE 

- VISUALLY COUPLED AIRBOURNE SYSTEMS SIMULATOR (VCASS) 

GOULD, INC. 

- INTERACTIVE CONTROL SYSTEMS 

LOS ALAMOS NATIONAL LABORATORY 

- ANTHROPOMORPHIC WELDING ROBOT 


These organizations have expertise in teleoperation, but are not primarily 
concerned with space operations. 

The Naval Ocean Systems Center (NOSC) in Hawaii has the most advanced 
telepresence system in operation of any of the organizations investigated by 
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the study team. Their telepresence unit has two arms, one 7 DOF and one 8 DOF, 
controlled by an cxoskeletal pair of arms worn by the operator. The system 
also has a mechanical spine and neck system, nearly able to duplicate human 
movement. The head contains a B&W stereo vision system and a binaural audio 
system. The teleoperator's head position and vision system are slaved to the 
operator's position by a head mounted stereo vision display using two B&W 
CRT's. Although the system is probably the most advanced in existence, the 
individual pieces are composed of off-the-shelf or old hardware designed for 
other purposes. Even with this handicap, the NOSC telepresence system is 
capable of many human motions and tasks. 

The Oak Ridge National Laboratory has embarked on a impressive program to 
develop a sealed nuclear laboratory in which all work will be performed by 
teleoperation. They are continuing to analyze the problem of task and work 
environment design, while at the same time entering a hardware development 
program so that the necessary technology will be available when needed. They 
are also performing an in depth study of workstations and human factors to 
develop a data base for future system design. Although much of the data will 
not apply to space telepresence due to different environmental constraints and 
lack of time delays, NASA could benefit from monitoring the results of this 
work, and particularly by adopting a similar approach to telepresence 
development. 

l.lf.S SUMMARY 

The facilities and expertise to develop telepresence technology for space 
use definitly exist. A serious effort will require participation by 
government, industry, and academia. 

Much of the hardware and software presently in existence are leftovers 
from the middle 1970 's, due to a drop in funding late in that decade. Many 
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research facilities and their equipment have suffered during the past few 
years. Renewed interest has helped this situation somewhat, but an effective 
development effort will require the updating of these research facilities. 

Military interest in this technology exists, but the scope and level of 
interest were not available to the study team. 

1.5 DEVELOPMENT PROGRAM 

1 . 5.1 INTRODUCTION 

In order to provide remote servicing operations during the early 1990 's, a 
telepresence development program must be started immediately. Much of the 
necessary technology already exists, but a significant development effort will 
be required to Integrate the technologies into an operational system, and space 
qualify the hardware. Section 1.5.2 presents the general development plan; 
section 1.5.3 presents potential and existing technologies, and suggests 
specific development goals. The technology development program is also useful 
for assessing the state-of-the-art and potential technologies, but is not 
intended to be exhastive. Automation technology is progressing extremely 
rapidly, and developments occur on a day to day basis. In addition, the 
competitiveness of the industry often limits dissemination of information about 
a new technology until it is nearing production and marketing. 

1 . 5.2 PROGRAM OUTLINE 

Figure I.I 4 presents the outline of a program which allows the 
evolutionary development of a space telepresence system. The first task, which 
should begin immediately, is the integration of the available technology into a 
ground demonstration system. This would allow the investigation of human 
factors and control system designs necessary for the development of an 


operational system. 
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Figure 1.14: DevelODtnent Program Outline. 



In parallel with the ground systems integration* an experiment performed 
in the shuttle middeck would be used to verify the manipulator control system 
for actual zero-g operations. Ground tests can simulate much of the effects of 
the space environment* but manipulation of small masses cannot be accurately 
simulated on the ground. Their mass and inertia are dominated by the mass and 
inertia of a ground simulator and the contact dynamics are extremely difficult 
to model on a computer. An experiment in the orb iter middeck would allow low 
mass manipulation tests in zero-g, without requiring the construction of a 
vacuum rated system. 

The results of the middeck experiment and the ground systems integration 
could be combined into a full scale demonstration and validation test on a 
pallet in the cargo bay. Other experiments onboard the orbiter could be 

I 

performed as necessary along with continuing ground technology development. 

All of these efforts lead to a 1990-1992 initial operational capability 
either for use on the TMS or as an attachment to the RMS for early operations. 
Continued systems development* most notably in software* and the addition of 
advanced technology when desirable, lead to a flexible and highly capable 
telepresence system. 

1 . 5.3 TECHNOLOGY DEVELOPMENT PROGRAM 

The technology development program discusses each of the six major 
technology areas. Each technology area has a listing of the key component 
technologies and divides them into three groups. Figure 1.15 shows the 
definitions of the three groups and how they are related to the NASA OAST seven 
technology readiness levels. The three groups also serve as a technology 
forecast. Group 1 is effectively current technology, Group II technology can 
be expected to be feasible in the next 10 years, and Group III technology will 
probably require at least 15 years to be developed. 
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The study group chose three levels Instead of attempting to divide the 
technology into seven levels for two reasons. First, three levels more clearly 
represent the actual condition of telepresence technology for a development 
program; second, because dividing this technology into seven levels often 
forced arbitrary decisions about the technology level, particularly for 
software development levels. 


NASA OAST SPACE SYSTEMS TECHNOLOGY MODEL (MAY 1980) 

TECHNOLOGY READINESS LEVELS 

LEVEL 1 BASIC PRINCIPLES OBSERVED AND REPORTED 
LEVEL 2 CONCEPTUAL DESIGN FORMULATED 

LEVEL 3 CONCEPTUAL DESIGN TESTED ANALYTICALLY OR EXPERIMENTALLY 
LEVEL 4 CRITICAL FUNCT I ON/CHARATER I ST I C DEMONSTRATION 
LEVEL 5 COMPONENT/BREADBOARD TESTED IN RELEVANT ENVIRONMENT 
LEVEL 6 PROTOTYPE/ENGINEERING MODEL TESTED IN RELEVANT ENVIRONMENT 
LEVEL 7 ENGINEERING MODEL TESTED IN SPACE 

TELEPRESENCE TECHNOLOGY GROUPS 
GROUP I (LEVELS 5"7) 

'TECHNOLOGY READY TO BE INTEGRATED INTO THE 
PALLET FLIGHT EXPERIMENT (1986). NOT NECESSARILY 
SPACE QUALIFIED 

GROUP I I (LEVELS 3“5) 

'SIMILAR TECHNOLOGY TO GROUP I, WITH 
INCREASED PERFORMANCE, BUT REQUIRING SOME 
DEVELOPMENT, CAN BE READY FOR IOC FLIGHT (1992) 

GROUP I I 1 (LEVELS I- 3 ) 

'BREAKTHROUGH IN TECHNOLOGY TO HIGH 
PERFORMANCE, REQUIRING BASIC RESEARCH 
PRIOR TO SYSTEM DEVELOPMENT 

FIGURE 1.15: TECHNOLOGY LEVELS AND GROUPS 


This presentation is very useful as an addition to the technology 
assessment of section 1.3. l^ut is more important as an indication that, for 
telepresence technology, systems integration is as important as the development 
of improved components. Most of the technology necessary for near term 
telepresence Is in Group I, and all of it is either in Group I or I I . A 
next to a technology item indicates that either it, or something very similar 
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to it should be developed for use in a near term telepresence system, or that 
it should be investigated further if it is a Group Ml technology. Items 
without a are included as reference points, and as indicators of technology 
aval lability. 

1 . 5 . 3-1 VISION 

GROUP 1 

-NOSC-TYPE STEREO BW CAMERAS, HELMET 
MOUNTED DISPLAYS 

GROUP I I 

-NOSC-TYPE VISION SYSTEM WITH COLOR* 

-VCASS WIDE-ANGLE STEREO DISPLAY* 

-MULT I -DOF CAMERA PLATFORMS* 

GROUP I I I 

-COMPUTER AUGMENTED IMAGES* 

-PREDICTIVE VIDEO DISPLAYS* 

-SCANNING LASER IMAGER 
-VARIABLE FOCAL PLANE 3-0 DISPLAY 
-HOLOGRAPHIC DISPLAYS 

Video camera and display techno.logy is advancing so rapidly that color 
vision systems could be available for a 1986 pallet experiment. The cameras 
and displays exist, but need to be integrated into an operational system. 

Development of computer augmented images and predictive displays has 
begun, but will require several years of effort before it can be considered for 
operational use. The development of these technologies ia not critical for 
telepresence, but should be very useful for advanced operations and reducing 
the effects of time delays. 

Laser imaging systems are useful in difficult lighting situations and for 
obtaining extremely accurate range, position, and velocity information about an 
object; it may be very useful during docking, but is of limited use for actual 
telepresence. 

3D vision technology is a very advanced ■ technology which may not have 
significant advantages over a stereo vision system. 
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1.5. 3. 2 MANIPULATORS 


GROUP I 

-ELECTRIC INDUSTRIAL ROBOT ARM 
-NUCLEAR INDUSTRY TELEOPERATORS 
-MIT BEAM ASSEMBLY TELEOPERATOR (BAT)* 
-MARTIN MARIETTA PFMA* 

GROUP I I 
-GRAPPLE ARMS* 

-INTERCHANGEABLE ARMS* 

-SECOND GENERATION SPACE MANIPULATOR* 
-NON-ANTHROPOMORPH I C ARM 

GROUP I I I 

-MODULAR MANIPULATOR* 

-TENDON ACTUATED MANIPULATOR* 

-FLEXIBLE MANIPULATOR (TENTACLE) 


The BAT and the PFMA are the two most advanced manipulators built for 
space operations or simulation. The BAT is designed for neutral buoyancy 
simulation of structural assembly, and consequently only has 5 DOF and is not 
sized for spacecraft servicing; it does possess an advanced control system 
designed for remote teleoperation and supervisory control. The PFMA is sized 
for spacecraft servicing, and was designed for near term space use; it has 7 
DOF and can be space qualified easily, but, at present, it lacks a useful 
teleoperation control system. It is currently controlled by computer, or by 
switch control of individual joint position. The merger of the technology 
involved in these two manipulator systems, plus the work which has been done at 
JPL, would produce a highly capable space specific telepresence manipulator. 

Grapple arms require development, but this should be a small effort, since 
most of the technology can be borrowed from previous manipulator development 
efforts. The same is true for interchangeable manipulator arms: work is needed 
to develop the necessary interface and modular construction technology, but 
most of what is needed can be taken from existing manipulator technology. The 
next generation (post PFMA) space manipulator should incorporate 
state-of-the-art technology to achieve the EVA equivalent capabilities 
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described in sections 1.2.2. 1 to 1.2. 2. 4 . It should have an advanced control 
system, a high power-to-weight ratio, interchangeable construction, and sizing 
based upon more up-to-date estimates of actual spacecraft designs and sizes. 

A non-anthropomorph i c manipulator is feasible in terms of actual 
manipulator construction, but the feasibility of the control and operation of 
such a device is uncertain. Humans have been able to control manipulators 
which were not the size or shape of a human arm, but a truly 

non-anthropomorph i c arm usually has a different number of DOF. A human trying 
to control a 10 DOF arm, for example, could easily have trouble specifying the 
positions of 3 of the 10 DOF because the human arm is essentially 7 DOF (the 
exact number depends upon how the DOF are defined, for example, the ability to 
"shrug" a shoulder may add a DOF, but may not be able to add any useful amount 
of control to a master-slave manipulator system). This technology is worthy of 
Investigation, but it is not a high priority item. 

The modular manipulator and the tendon actuated manipulator both have 
significant potential benefits, and should be thoroughly investigated. 

A flexible or tentacular manipulator is the non-anthropomorph I c 
manipulator taken to an extreme, and suffers from the same problems. It may 
have to wait for complete computer control of the manipulator system. 

1 . 5 . 3. 3 end effectors 

GROUP I 

-1 DOF GRIPPERS 
-SIMPLE GRAPPLES (RMS)* 

-SIMPLE TOOL (AND/OR END EFFECTOR) INTERCHANGE MECHANISMS* 

-DOCKING DEVICES* 

GROUP I I 

-GENERALIZED GRIPPERS* 

-MULTI -DOF GRIPPERS* 

-ADVANCED TOOL (AND/OR END EFFECTOR) INTERCHANGE MECHANISMS* 

GROUP I I I 
-HAND ANALOGUES* 
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The 1 DOF gripper is simply a pincher similar to a lobster claw and is 
capable only of simple opening/closing motion. Some generalized grippers may 
also be only 1 OOF, but will use a compliant surface to improve their gripping 
capability. There are a number of generalized gripper designs; this 
particular technology has items in both Group I and Group II technologies. 

Tool interchange mechanisms have been demonstrated in the laboratory and 
used in simulations. Some of this technology could be used today, but an 
effective system requires further development. 

Docking devices also require more development, but this Is a far more 
mature technology than tool interchange mechanisms, and is consequently a lower 
priority. 

Multi -DOF grippers, and some advanced tools or end effectors, begin to 
blur the distinction between tools and mechanical hands. All of these 
technologies should receive the highest research priority because of their 
potential to significantly increase the dexterity of telepresence systems. 


1 . 5 . 3. it SENSORS 


GROUP I 

-PROXIMITY SENSORS* 

-MOTOR CURRENT FORCE AND TORQUE SENSORS 
-ARM POSITION SENSORS* 

-SELSPOT* 

-LASER BAR-CODE READERS 

GROUP I I 
-SLIP SENSORS* 

-STRAIN GAUGE FORCE AND TORQUE SENSORS 
-FIBEROPTIC FORCE AND TORQUE SENSORS* 
-LASER RANGE AND POSITION SENSORS* 

GROUP I I I 
-TACTILE SENSORS* 

-ACOUSTIC STRUCTURAL ANALYZERS 


Proximity sensors are a mature technology for many applications, and a RMS 
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end effector using proximity sensors has been developed and tested at JPL. 

Motor current sensors have been used for many years by the nuclear 
industry to provide force data for manipulator control, and is a mature 
technology. Strain gauge sensors are comparatively new to this application, 
but they are also a well developed technology. Fiberoptic current and strain 
sensors appear to be superior to standard current and strain sensors (section 
1.3*2. 5*2), and should be developed further. 

Slip sensors are very useful for automatic grip control and prevention of 
excessive loads (if slip occurs, it is often due to the application of 
excessive force). JPL has demonstrated slip sensors in the laboratory. 

Arm position sensors, such as potentiometers and optical encoders which 
measure joint position, are in use today for terrestrial applications. The 
Selspot is an infrared device that measures arm position directly, and is 
useful when precise measurements are necessary. All of these sensors are 
within present technology, and require minimal development prior to space 
qual i f i cation. 

The laser bar-code reader, similar to those used in supermarket checkout 
stands, is potentially useful for aiding computers to recognize images. It is 
not necessary for near term telepresence unless research indicates that the 
operator needs to have objects in his field of view identified by computer. 

Unlike the code-reader, laser radar and ranging devices could be very 
useful for docking and operations in crowded worksites. Additional development 
of this technology is necessary; although demonstration units exist which 
produce very precise measurements at high speed. 

Tactile sensors are useful when tactile feedback is feasible, and could be 
used by advanced computer systems capable of understanding tactile data. This 
technology deserves further investigation because of its potential benefit. 

Acoustic signals transmitted through a structure are reflected and 
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modified by flaws and failures in the structure, and can potentially be used to 
analyze the structure. Although It may be useful for the assembly of large 
structures, it is not a priority research item. 

1.5. 3 . 5 COmUNICATIONS 
GROUP I 

-S BAND LINKS (TDRSS) 

-DATA COrtPRESSION AND ENCODING 

GROUP I I 
-K BAND LINKS* 

-ADVANCED DATA COMPRESSION AND ENCODING* 

GROUP I I I 

-SECOND GENERATION TDRSS 
-SPACE STATION CONTROL CENTER* 

-OPTICAL LINKS* 

-DIRECT LINKS (VIA NETWORK) 

The K band communications links require some hardware development for the 
spacecraft side of the link, but this is a strai ghtforeward design and test 
problem which does not require any significant advances in communications 
technology . 

Data compression is not required for a near term telepresence system using 
the K band links, but error correcting codes are desirable. As the system 
grows in complexity, other users compete for TDRSS time, and other 
communications links become available; data compression may become necessary. 
This technology is desirable and should be investigated, but is not an 
immediate priority. 

A second generation of TDRSS type spacecraft, or commercial communications 
satellites, will probably become available a few years after the deployment of 
a telepresence system. They are a desirable alternative to the TDRSS system, 
and will be developed without requiring involvement by telepresence 
researchers . 
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A space station control center offers the elimination of transmission time 
delays for work near the station. Since time delays reduce the capabilities of 
the system, this could be very important for delicate repair or assembly 
on-orb 1 1 . 

The investigation of optical communications could have numerous benefits. 

A laser Is more resistant to local interference and can carry far more 
information than radio transmissions, thus reducing the chances of 
communications being unavailable. Also, a telepresence system capable of 
optical communications would be readily able to take advantage of a ground and 
space network using fiberoptics and laser technology. A versatile optical 
network could automatically use the least time delay communications link. As 
ground stations become more common, the need for geosynchronous relays could be 
reduced for many operations, since the number of optical ground stations would 
permit direct communications with the spacecraft. Such a system could not be 
in place until post 2000, but Its potential capabilities deserve attention by 
NASA for telepresence, and any other activity involving communications. 


1 . 5 . 3. 6 CONTROL 


GROUP I 

-PILOTED DOCKING 
-MASTER-SLAVE MANIPULATOR* 

-MASTER-SLAVE MANIPULATOR WITH FORCE FEEDBACK* 
-NOSC TYPE CAMERA CONTROLLER* 

GROUP I I 

-REMOTELY PILOTED DOCKING 
-AUTOMATED DOCKING 
-MANIPULATOR WITH FORCE CONTROL* 

-VOICE CONTROL SYSTEM* 

-EVENT DRIVEN FUNCTIONS* 

-SUPERVISORY CONTROL* 

-FAIL SAFE CONTROL* 

GROUP I I I 

-CAMERA CONTROL BY EYE FOCUS* 

-ADVANCED SUPERVISORY CONTROL* 

-ADAPTIVE CONTROL* 
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Docking is not necessarily a telepresence function, but it is a necessary 
part of most telepresence missions. The US has a wealth of experience with 
piloted docking operations from the manned space program, but no experience 
with actual remotely piloted or automated docking operations. Both are 
desirable for telepresence because of time delays and the need to provide 
backup docking modes. Much development and testing remains before either mode 
is ready for use, but the problems should not be insurmountable: the Soviets 
have demonstrated both docking modes repeatedly. 

Various mastei — slave and multi-DOF hand controllers have been developed. 
The remaining work is to test the various options and select the best features 
for a telepresence controller, and to integrate the technology into a working 
system. Also, more attention must be paid to force control systems, since 
force feedback is not desirable in the presence of time delays. 

Camera position control by slaving to the operator's head orientation has 
been demonstrated, and can easily be applied to space telepresence. It is 
possible to monitor eye position and focus length to determine where a subject 
is looking; this is a possible method of controlling camera focus and possibly 
20om settings. If feasible, this type of system could greatly enhance an 
operator's sense of presence at the worksite, and it is worthy of further 
i nvest i gat i on . 

Voice control systems have been demonstrated for a variety of purposes and 
could improve the performance of a telepresence system. Little development 
work is necessary, but command modes and procedures must be selected and 
integrated into an operational telepresence system. 

Event driven displays have been tested at JPL, and have been found to be 
useful. A similar, but more advanced, use of this technology is to switch 
control modes, procedures, menus of instructions, etc. based upon some 
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predetermined cue. Event driven functions could help an operator to handle 
complex tasks and procedures. This technology is not well developed, and its 
interaction with an operator requires a serious research effort in a simulated 
work environment. 

Supervisory control has been demonstrated in the laboratory, and could 
become an important part of a telepresence system. It is also the first step 
towards autonomous systems. Supervisory control, and the more autonomous 
advanced modes, should be investigated thoroughly. 

An important technology which has received little attention so far is a 
fail safe control system. If an operator makes an obvious mistake or 
communications are lost, the onboard computers should be able to prevent damage 
to the telepresence system or the spacecraft It is working on. Such a control 
system would need some very advanced capabilities to be completely fail safe, 
but much can be accomplished with near term technology, while research into 
Group III technologies provides more capable systems. 

Adaptive control can potentially enable telepresence systems to respond 
autonomously to anomalous situations of varying severity (amount of excursion 
from nominal performance), and to learn from past experience. Adaptive control 
is necessary for some of the advanced modes of supervisory control. It is an 
important, and potentially very useful, technology that will require much 
research before it is ready for use, 

1 . 5 . A DEVELOPMENT PROGRAM SUMMARY 

The development of a telepresence system capable of achieving the same 
results as EVA can be completed by 1992. An examination of the technology 
assessment (section I. 3 ) and the preceeding development program show that all 
of the necessary technology either exists or has been demonstrated in the 
laboratory. Several areas require individual technology development efforts. 
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but the majority of a development program must focus on the integration of the 
telepresence system. This requires the ability to simulate the workspace, 
operate a ground telepresence system, and test the control system interaction 
with a human operator. 

In order to complete an operational telepresence system by 1932, a 
development program must begin immediately. NASA needs to more clearly define 
which space missions are going to use teleoperated servicing, and to build 
spacecraft with servicing as a design criteria. Industry should be used to 
develop the necessary space qualified hardware and build the telepresence 
system. Academic institutions should be involved to determine control system 
and operator Interactions, to identify the proper uses of the various 
technologies available for telepresence, and to begin or continue the 
development of the Group III technologies. Since the capabilities and 
expertise of NASA, industry, and academic institutions often overlap, and 
because each type of organ! 2 ati on approaches the problem from a different 
perspective, each should participate in all phases of the development effort. 
The actual hardware necessary for a ground telepresence development system need 
not be very expensive, so NASA should encourage in-house, industrial, and 
academic ground development systems. 

A ground development program, coupled with space experiments as necessary, 
will provide NASA with a highly capable and versatile teleoperation system able 
to meet both near and long term needs. 

1.6 TELEPRESENCE TECHNOLOGY BASE DEVELOPMENT CONCLUSIONS 
1.6.1 TELEPRESENCE IS NEEDED 

Future NASA plans, both short and long term, call for spacecraft 
servicing, structural assembly, and contingency operations. The success of 
large scale space operations, both for NASA and industry, will require the 
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capability to perform versatile operations in space, similar to those 
associated with any large program on the ground. 

Telepresence is well suited to this demanding work environment because it 
provides both the ability to use human judgment and manipulative skill, and 
the ability to use autonomous technology (robotics) when it becomes available. 
Thus, telepresence has the advantages of both machine and human capabilities. 

Due to the nature of near term spacecraft design, and the specifics of 
feasible near term technology (system deployment by 1992), the initial 
telepresence system should be designed to be capable of accomplishing the same 
tasks as an astronaut in a pressure suit (present EVA suit technology, see 
section 1 .2.2.5) • 

The lack of definite long term plans, and the rapid advance of electronics 
and control technology, make determination of specific long term telepresence 
objectives difficult. Since artificial intelligence and manipulator technology 
will continue to advance, as will the demands placed upon remote servicing 
systems, it is reasonable to conclude that long term telepresence systems will 
be capable of very complex mechanical tasks and high levels of autonomy. 

1.6.2 TELEPRESENCE IS FEASIBLE 

Most of the necessary technology for an EVA equivalent telepresence system 
has already been developed. Certain areas, such as vision systems, need 
development of specific components, such as small, lightweight color displays, 
but these areas are usually being developed independent of NASA. Space 
adaptation and qualification of these technologies is also necessary, but the 
most important task is system integration. During this process, human operator 
interactions with the hardware and the control system must be analyzed to 
permit design of the actual flight system. 

Table 1.3 presents a summary of the technology requirements for a near 
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- STEREO-OPTIC VISION SYSTEM— PREFERABLY COLOR- 
CAPABILITY TO SLAVE TO OPERATOR'S HEAD POSITION 

- HEAD-MOUNTED VISION DISPLAY SYSTEM 

- TWO 7 DOF MANIPULATOR ARMS WITH FORCE CONTROL 

- TWO GRAPPLE ARMS OR ONE DOCKING DEVICE 

- INTERCHANGEABLE END-EFFECTORS 

- OPERATOR USES FORCE- I ND 1 CAT I NG HAND CONTROLLERS 
OR EXOSKELETAL ARMS FOR CONTROL 

TABLE 1.3: TECHNOLOGY REQUIREMENTS SUMMARY 

term telepresence system. The study group believes that such a system could be 
built and flown by 1992 if a laboratory development program is begun 
immediately. Telepresence technology, and the research centers involved with 
it, have been adversely affected by a lack of funding during the past few 
years, but the technology, facilities, and personnel necessary for the 
development of a telepresence system are available. 
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